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Abstract 

The  Vehicular  Simultaneous  EMI  and  Magnetometer  System  (VSEMS),  developed  under 
ESTCP  project  MM-0208,  was  deployed  at  the  former  Kirtland  Bombing  and  Gunnery  Range 
during  the  fall  of  2006  as  part  of  the  ESTCP  Wide  Area  Assessment  Pilot  Program. 
Magnetometer  and  EM61  Mk2  data  were  concurrently  acquired  on  pre-planned  transects 
(specified  by  another  contractor  running  Visual  Sampling  Plan  on  the  basis  of  existing  data  from 
a  Conceptual  Site  Model)  to  bound  the  extent  of  known  bombing  targets  and  assist  in  discovery 
of  new  ones.  Both  magnetometer  data  and  EM61  data  were  examined  overnight  to  extract 
visually-derived  information  on  whether  the  transects  appeared  to  run  over  a  bombing  target. 
Magnetometer  data  were  analyzed  overnight  to  provide  inversion  results  (location  and  depth 
estimates)  for  objects  of  interest  in  the  data.  These  data  were  used  by  another  contractor,  and  by 
the  ESTCP  program  office,  to  refine  the  CSM  and  to  specify  new  transects.  In  addition  to 
transects,  high-resolution  100%  geophysical  survey  data  were  also  acquired  in  areas  located 
radially  outward  from  bombing  targets  to  determine  anomaly  fall-off,  and  in  areas  that  were 
flagged  as  being  of  interest  after  interrogation  of  data  from  the  airborne  sensing  layers.  193  acres 
of  transects  and  159  acres  of  100%  geophysical  survey  areas  were  acquired,  for  a  total  of  352 
acres.  A  total  of  9969  anomalies  were  extracted  from  the  data. 

1  Introduction 

1.1  Background 

One  of  the  recommendations  of  the  2003  Defense  Sciences  Board  (DSB)  Report  on  Unexploded 
Ordnance  was  to  immediately  assess  the  scope  of  ordnance  contamination  of  roughly  ten  million 
acres  of  land  on  FUD  and  BRAC  sites  and  rapidly  ascertain  what  percentage  of  this  acreage 
actually  contains  UXO.  Airborne  technologies  are  well-suited  for  acquiring  data  over  sites 
comprising  thousands  of  acres  and  assessing  the  degree  of  UXO  contamination.  High-altitude 
fixed-wing  airborne  visual  and  LIDAR  technologies  have  been  shown  to  be  effective  in  detecting 
ordnance-related  features,  and  helicopter-based  magnetometry  has  been  shown  to  be  effective  in 
detection  of  individual  UXO  objects  of  a  range  of  sizes.  However,  because  magnetic  field 
strength  falls  off  as  one  over  the  cube  of  the  distance  (1/R3),  the  increased  sensor  height  of 
helicopter-based  sensors  makes  detection  of  objects  smaller  than  60mm  very  difficult.  Pulsed 
induction  sensors  such  as  the  Geonics  EM61,  frequently  the  sensor  of  choice  for  ground-based 
UXO  detection,  have  an  even  steeper  (1/R6)  falloff.  Further,  there  are  limits  to  the  safe  terrain¬ 
following  of  helicopter-based  systems.  For  these  reasons,  ground-based  DGM  systems  have  a 
role  to  play  in  wide  area  assessment,  both  for  close-in  detection  of  object  boundaries,  as  well  as 
in  validation  and  verification  of  the  results  from  airborne  surveys.  The  technology  used  for  this 
project  -  the  Vehicular  Simultaneous  EMI  and  Magnetometer  System  (VSEMS  -  formerly 
known  as  the  Simultaneous  Multisensor  STOLS)  has  the  added  benefit  of  collecting  total  field 
magnetometer  and  EM61  data  simultaneously  in  a  single  survey  pass,  thereby  acquiring  data 
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from  the  two  sensors  most  widely  used  for  UXO  detection  and  most  widely  accepted  by 
regulators.  For  Wide  Area  Assessment,  the  two  sensors  can  be  useful  in  a)  detecting  different 
objects,  b)  providing  redundant  data  in  the  event  of  unanticipated  noise  or  clutter  situations,  and 
c)  helping  to  screen  out  geology. 

1.2  Objectives  of  the  Demonstration 

The  stated  objectives  of  the  demonstration  as  listed  in  the  Demonstration  Plan  were: 

•  To  use  VSEMS  to  collect  magnetometer  and  EM61  DGM  data  on  pre-planned  transects 
(generated  by  another  contractor  running  Visual  Sample  Plan  (VSP)  transect-planning 
software)  so  that  the  WAA  project  as  a  whole  could  use  these  data  to  refine  bombing 
target  locations,  extents,  and  edges  that  were  already  approximately  known  from 
conceptual  site  models. 

•  To  visually  inspect  and  interpret  the  transect-based  data  for  the  presence  of  bombing 
target  edges  or  extent,  or  evidence  of  other  subsurface  entities  such  as  trenches  and  pits, 
that  might  be  of  interest  in  a  WAA  context. 

•  To  extract  anomaly  locations  and  size  estimates  from  the  transect  data  and  generate 
anomaly  lists  which,  in  turn,  were  fed  to  VSP  to  design  further  sets  of  transects. 

•  To  use  VSEMS  to  collect  and  analyze  full-coverage  data  over  selected  areas  for 
validation  purposes,  extracting  anomaly  locations  and  depth  and  size  estimates  from  the 
surveyed  areas. 

•  To  visually  inspect  and  interpret  the  full-coverage  data  for  the  presence  of  bombing  target 
edges  or  extent,  or  evidence  of  other  subsurface  entities  such  as  trenches  and  pits,  that 
might  be  of  interest  in  a  WAA  context. 

•  To  demonstrate  and  validate  the  use  of  VSEMS,  which  had  been  further  improved  over 
prior  fieldings  with  the  installation  of  a  new  EM61  Mk2  array,  as  a  viable  survey-ready 
DGM  tool  that  consistently  and  reliably  generates  very  high-quality  concurrent  EM61 
and  magnetometer  data. 

All  of  the  above  objectives  were  met.  The  system  collected  over  350  acres  of  concurrent 
magnetometer  and  EM61  Mk2  data  at  the  Former  Kirtland  Bombing  and  Gunnery  range,  and 
these  data  were  an  asset  to  the  Wide  Area  Assessment  Pilot  Program. 

1.3  Regulatory  Drivers 

The  Wide  Area  Assessment  Pilot  Program  itself  was  driven  by  congressionally-mandated  desire 
to  shrink  the  footprint  of  potentially  UXO-contaminated  land  as  per  the  DSB  report.  Because  the 
technology  involves  combining  the  two  sensors  most  validated  against  UXO  for  digital 
geophysical  mapping  -  total  field  magnetometers  and  EM61  pulsed  induction  coils  -  there  are  no 
specific  regulatory  issues  above  those  that  apply  to  all  DGM  data. 

1.4  Stakeholder  /  End-User  Issues 

Because  the  technology  involves  combining  the  two  sensors  most  validated  against  UXO  for 
digital  geophysical  mapping  -  total  field  magnetometers  and  EM61  pulsed  induction  coils  - 
there  are  no  specific  end-user  issues  above  those  that  apply  to  all  DGM  data. 
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2  Technology  Description 


2.1  Technology  Description  and  Application 
Overview 

The  VSEMS  used  on  this  project  was  originally  developed  by  SAIC  with  support  from  ESTCP 
under  project  MM-0208,  implemented  through  a  CRADA  between  SAIC  and  the  US  Army 
Corps  of  Engineers  Huntsville  Center  (CEHNC).  VSEMS  is  the  only  system  in  the  world  that 
simultaneously  collects  high-quality  data  from  COTS  total  field  magnetometers  and  COTS 
EM61  Mk2  pulsed  induction  sensors  on  a  single  towed  platform.  It  substantially  leveraged  GEO¬ 
CENTERS’  (now  SAIC)  existing  “STOLS”  GPS-integrated  towed  magnetometer  array  as  a 
development  platform,  and  augmented  it  with  newly  designed  interleaving  hardware,  a  new 
proof-of-concept  non-metallic  towed  platform,  and  existing  EM61  electronics  and  coils.  Further 
development  which  integrated  modem  Geonics  EM61  Mk2  hardware  was  funded  by  the  US 
Army  EQT  program  and  by  ATC.  Because  portions  of  the  system  are  13  years  old  whereas  other 
portions  were  only  designed  to  survive  a  single  2002  APG  fielding,  the  system  is  best  operated 
by  its  inventors.  Through  continuous  incremental  improvement  and  careful  operation,  the  system 
has  proven  itself  capable  of  reliably  prosecuting  large  geophysical  surveys. 

Theory  of  Operation 

Historically,  simultaneous  deployment  of  magnetometers  and  pulsed  EM  such  as  the  Geonics 
EM61  on  a  common  platform  has  not  been  possible  due  to  the  fact  that  the  EM  transmission 
pulse  is  asynchronous  with  the  magnetometer  sampling,  and  thus  is  picked  up  by  the 
magnetometers  as  noise.  Even  at  10  feet  -  a  practical  separation  distance  for  sensor  co-location 
on  a  common  towed  platform  -  EM61 -induced  noise  is  over  100  nT,  rendering  concurrently- 
collected  magnetometer  data  useless. 

Under  ESTCP  Project  MM-0208,  SAIC  developed  hardware  that  monitors  the  pulse  from  the 
EM61,  waits  a  preset  amount  of  time  for  the  pulse  and  the  secondary  fields  generated  by  the 
pulse  to  ring  down,  then  samples  the  magnetometer  for  a  short  window.  The  newly-developed 
MPC  board  is  designed  to  interleave  the  magnetometer  and  EM61  data  acquisition  cycles  as 
follows.  The  MPC  circuitry  looks  for  the  1  Pulse  Per  Second  (PPS)  from  the  GPS,  then  looks  for 
the  rising  edge  of  the  next  EM61  transmission  pulse.  The  system  timing  then  uses  a 
programmable  waiting  period  and  a  sampling  period.  The  75  Hz  EM61  transmission  pulse  comes 
in  every  13.3  ms.  The  board  waits  8  ms,  at  which  point  the  EM61  transmission  pulse  has  died  off 
(this  has  been  verified  by  direct  measurement).  The  MPC  board  then  samples  the  magnetometers 
for  5  ms,  during  the  period  in  which  the  EM61s  are  not  transmitting.  In  this  way,  the 
magnetometers  are  only  sampled  when  the  EM61s  are  quiet.  The  timing  diagram  for  this 
interleaved  synchronous  data  acquisition  is  shown  in  figure  1.  Note  that  in  this  new  design, 
acquisition  of  magnetometer  data  is  triggered  by  the  receipt  of  a  75  Hz  strobe  from  the  EM61 
electronics  after  the  GPS’  1  PPS. 
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Figure  1:  Conceptual  Timing  Diagram  of  Synchronous  EM61  and  Magnetometer  Data  Acquisition.  Note  that 
magnetometer  sampling  only  occurs  when  EM61  transmission  pulse  has  died  down. 


Key  Design  Criteria 

In  addition  to  the  interleaving  electronics  and  software  that  sample  the  magnetometers  after  the 
secondary  field  induced  by  the  EM61  pulse  has  rung  down,  the  total  system  design  that  hosts 
both  the  magnetometers  and  the  EM61s  in  a  low-noise  environment,  utilizing  a  low- ferrous 
vehicle  and  a  non-metallic  towed  platform,  is  a  key  design  factor. 

Schematics,  Figures,  and  Layout 

The  timing  diagram  for  synchronous  data  acquisition  is  shown  in  the  figure  above.  The  system, 
showing  the  major  components  (low-ferrous  vehicle,  non-metallic  platform,  GPS,  magnetometer 
array,  and  EM61  array),  is  shown  below  in  figure  2. 


Figure  2:  Layout  of  Major  VSEMS  Components 


2.2  Previous  Testing  of  the  Technology 

The  VSEMS  technology  is  well-tested  as  described  in  the  final  reports  for  project  MM-0208. 
Additional  improvements  and  deployments  of  the  technology  include: 

•  2003.  Incremental  improvements  funded  by  CEHNC  (platform  reinforcement). 
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•  2003.  VSEMS  surveys  100  acres  at  the  Former  Lowry  Bombing  and  Gunnery  Range  in 
Aurora,  CO. 

•  2003.  Incremental  improvements  funded  by  CEHNC  (additional  EM61  channels,  larger 
EM61  coils,  rugged  computer,  and  additional  platform  reinforcement). 

•  2003.  VSEMS  surveys  100  acres  at  Portland  International  Airport. 

•  2004.  VSEMS  surveys  100  acres  at  a  mid- Atlantic  housing  site. 

•  2004.  Incremental  improvements  funded  by  EQT  (new  EM61  Mk2  array). 

•  2005.  VSEMS  surveys  APG  and  YPG  Standardized  UXO  Test  Sites. 

2.3  Factors  Affecting  Cost  and  Performance 

The  economics  of  surveys  bid  at  a  fixed  acreage  rate  per  day  depend  on  the  coverage  rate. 
Smooth,  grassy  areas  that  have  already  been  run  over  by  heavy  equipment  are  far  more 
vehicularly  navigable  than  rocky  or  stumpy  areas,  and  resulting  lower  coverage  rates  engender 
higher  survey  cost.  This  is  particularly  true  due  to  the  proof-of-concept  nature  of  the  fiberglass 
towed  platform,  which  has  exceed  all  expectations  but  still  must  be  treated  gently. 

Any  activity  that  lessens  the  percentage  of  the  survey  day  actually  spent  collecting  data 
dramatically  cuts  into  survey  efficiency.  These  include:  daily  drive  to  the  survey  site;  mandated 
breaks  for  UXO  techs;  mandated  daily  QC  procedures;  time  spent  locating  site  boundaries,  grid 
squares,  and  transect  endpoints;  and  downtime  due  to  equipment  malfunction  and 
troubleshooting.  These  things  are  not  unique  to  VSEMS  but  should  not  be  overlooked. 

Because  VSEMS  is  a  vehicular- towed  array,  it  requires  a  tractor  /  trailer  to  transport  it,  which  is 
more  costly  than  using  a  man-portable  system  that  can  be  cheaply  shipped.  As  such,  driving 
services  must  be  contracted. 

At  the  Kirtland  site,  the  following  factors,  in  approximate  decreasing  order  of  priority,  affected 
productivity: 

•  GPS  issues  stemming  from  the  size  of  the  site  (the  distance  between  GPS  base  and  rover), 
and  interference  in  the  radio  link  between  base  and  rover  stemming  from  the  high  amount 
of  construction  activity  in  Albuquerque  using  GPS 

•  Learning  curve  in  first  inputting,  then  following,  pre-planned  transects 

•  Switching  between  transect  surveys  and  100%  geophysical  surveys  (driving  to,  then 
locating,  the  comers  of  the  100%  geophysical  survey  areas) 

•  Downtime  due  to  mechanical  and  electrical  issues 

•  Limitation  of  forward  rate  of  advance  due  to  roughness  of  terrain  and  fragility  of 
prototype  fiberglass  towed  platform 

•  Slow  forward  rate  of  advance  needed  to  maintain  planned  down-track  EM61  data 
spacing,  particularly  in  1 00%  geophysical  survey  areas 

•  Fences  interfering  with  planned  transect  lines 

•  Weather 

•  Geometry  of  100%  geophysical  survey  areas  (short  lines  requiring  many  turn-arounds) 

Individually,  the  COTS  total  field  magnetometers  and  EM61  Mk2  sensors  used  by  VSEMS  have 
well-documented  performance  envelopes  that  are  not  altered  by  their  presence  in  VSEMS.  As 
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such,  the  system’s  performance  has  the  sensors’  fundamental  limitations  of  sensitivity;  the 
magnetometer  signal  falls  off  as  1/R3;  the  EM61’s  as  1/R6.  Magnetometers  are  sensitive  to  the 
presence  of  both  ferrous  cultural  objects  as  well  as  iron-bearing  soils,  and  thus  their  usefulness  is 
limited  on  sites  with  dense  building  configurations  or  complex  geologies.  EM61s  are  more 
immune  to  these  things,  but  this  is  partially  due  to  their  inherent  decreased  sensitivity,  and  as 
such  they  will  not  detect  deep  UXO  objects  as  well  as  magnetometers. 

2.4  Advantages  and  Limitations  of  the  Technology 

The  complimentary  nature  of  the  weaknesses  of  the  sensors  described  above  is  one  of  the  very 
things  driving  their  concurrent  use  in  YSEMS.  The  overriding  advantage  of  the  technology  is  the 
ability  to  concurrently  collect  both  magnetometer  and  EM61  data  in  a  single  survey  pass  and 
thus  compensate  for  each  other’s  shortcomings.  MTADS,  in  both  its  NRL  and  Blackhawk- 
fielded  configurations,  has  a  separate  towed  magnetometer  and  towed  EM6 1  platform,  and  thus 
requires  two  separate  surveys  to  acquire  both  mag  and  EM  data  sets.  Further,  since  VSEMS’ 
sensors  are  mounted  on  a  common  rigid  sensor  platform,  this  all  but  ensures  that  the  two  sensors 
will  run  over  the  same  ground  locations,  and  thus  essentially  eliminates  another  problem  of 
performing  two  separate  surveys  -  that  the  data  acquired  in  separate  survey  passes  may  not 
traverse  the  same  objects  in  the  same  way,  if  at  all,  which  may  limit  the  efficacy  of  the  data  for 
discrimination  algorithms.  Note  that,  in  descriptions  of  data  processing  below,  when  we  refer  to 
the  data  being  processed  on  a  10  cm  grid,  this  refers  to  the  use  of  in-house  software  to  visualize 
the  magnetometer  and  EM61  data  and  to  invert  the  magnetometer  data.  However,  when  the 
magnetometer  and  EM61  data  sets  are  each  geo  located  and  processed  and  written  out  in  an 
ASCII  format  for  importation  into  Geosoft  Oasis  Montaj™,  there  is  no  10  cm  quantization  of  the 
geolocation  data. 

The  main  limitations  of  the  technology  as  compared  to  MTADS  are  that  the  cross-track 
magnetometer  spacing  in  MTADS  is  tighter  than  VSEMS  (1/4  meter  versus  14  meter).  The 
MTADS  sensor  platforms  are  instrumented  to  measure  pitch  and  roll,  and  their  data  processing 
software  uses  these  data  to  more  accurately  position  sensor  updates.  However,  note  that  these  are 
limitations  on  the  specific  implementation  of  the  technology  as  manifested  in  the  current 
VSEMS.  The  core  technology  -  interleaving  acquisition  of  magnetometer  data  between  EM61 
pulses  -  does  not  have  these  limitations.  The  main  limitation  of  the  core  interleaving  technology 
is  that  it  applies  only  to  pulsed  induction  EM  systems,  and  is  not  applicable  to  frequency-domain 
EM  systems.  There  are  other  competing  technologies  for  concurrent  magnetometry  and  EM  (G- 
Tech,  AETC),  but  as  of  this  date,  none  of  them  use  a  commercial-off-the-shelf  industry- standard 
EM61,  and  none  of  them  are  conducting  real-world  350-acre  surveys. 

There  is  an  argument  that  the  forward  rate  of  advance  of  VSEMS  is  limited  by  the  EM61’s 
slower  update  rate  (10  Hz  as  compared  to  75  Hz  for  the  magnetometer  update  rate),  and  thus  that 
the  concurrent  use  of  mags  and  EM61s  is  inherently  less  productive  than  magnetometers  alone. 
We  feel  that,  if  all  factors  were  equal,  this  would  be  true,  but  that  all  factors  are  almost  never 
equal.  For  example,  at  the  Kirtland  survey,  we  could  not  have  driven  much  faster  than  we 
already  did  while  towing  our  prototype  fiberglass  platform.  On  extremely  hospitable  (e.g.,  soccer 
field)  topography,  the  concurrent  use  of  EM61s  will  limit  speed,  but  this  should  be  taken  in  the 
context  of  the  list  of  other  factors  above. 
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3  Demonstration  Design 


3.1  Performance  Objectives 

The  following  were  the  primary  performance  objectives  from  the  demonstration  test  plan.  These 
were  provided  by  the  ESTCP  Program  Office  for  both  of  the  ground-based  systems  on  the  Wide 
Area  Assessment  Pilot  Program  (e.g.,  MTADS  and  VSEMS). 


Table  1:  Primary  Transect  Performance  Objectives/Metrics  and  Confirmation  Methods  Relating  to  Detection 

of  Target  Areas  and  Target-Free  Areas 


Type  of 
Performance 
Objective 

Performance 

Criteria 

Expected 

Performance 

(Metric) 

Performance 
Confirmation  Method 

Actual  Performance 
Objective  Met? 

Qualitative 

Reliability 

and 

Robustness 

General 

Observations 

Operator  feedback  and 
recording  of  system 
downtime  (length  and 
cause) 

System  was  Reliable 

Yes 

Terrain  / 

Vegetation 

Restrictions 

General 

Observations 

Correlation  of  areas  not 
surveyed  to  available 
data  (topographical 
maps,  etc.) 

System  surveyed 
vehicularly  navigable 
areas 

Yes 

Quantitative 

Survey  Rate 

12.5  acres  /  day 

Calculated  from  survey 
results 

9.5  acres /day 

No 

Data 

throughput 

All  data  from  day 
x  processed  for 
anomalies  and 
submitted  by  end 
of  day  x+1 

Analysis  of  records  kept 
/  log  files  generated 
while  in  the  field 

Analyzed  Anomalies 
Submitted  by  Next  Day 
Yes 

Percentage 
of  Assigned 
Coverage 
Completed 

>95%  as  allowed 
by  topography 

Calculated  from  survey 
results 

All  Transects  Completed 
Yes 

75%  of  100% 
geophysical  survey 
Areas  Completed 

No 

Transect 

Location 

95%  within  2 
meters  of 
requested 
transects 

Calculated  from  survey 
results 

Most  Transects  Aligned 
Yes 

These  are  discussed  fully  in  section  4  below. 

The  following  were  the  secondary  performance  objectives  from  the  demonstration  test  plan, 
again,  as  provided  by  the  ESTCP  Program  Office. 
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Table  2:  Secondary  Transect  Performance  Objectives/Metrics  and  Confirmation  Methods  Relating  to 

Characterization  of  Target  Areas 


Type  of 
Performance 
Objective 

Performance 

Criteria 

Expected 

Performance 

(Metric) 

Performance 
Confirmation  Method 

Actual  Performance 
Objective  Met? 

Qualitative 

Ability  of 
Analyst  to 
Visualize 
Targets  from 
Survey  Data 

All  targets  in 
survey  area 
identified 

Data  Analyst  feedback 
and  comparison  to  100 
geophysical  survey  data 
/  other  demonstrators 
results 

Targets  Readily 
Visualizeable 

Yes 

Quantitative 

Location  of 

Inverted 

Anomalies 

<  0.15  m 
horizontal 
<30%  vertical 

Comparison  to  Test  Strip 
Ground  Truth 

0.13  m  horizontal 

0.20  m  vertical 

Yes 

Probability 
of  False 

Alarm 

<5%  of  identified 
anomalies 
correspond  to  no 
ferrous  metal 
source 

Validation  Sampling 
(100%  survey)  and/or 
Remediation  Sampling 
(digging) 

13.5%  correspond  to 
apparent  geology 

No 

Signal  to 

Noise  Ratio 
(SNR)  for 
Calibration 
Objects 

+/- 10%  of 
expected  from 
Standardized 
UXO  Technology 
Demonstration 
Site  Performance 

Comparison  of 
Calibration  Objects 
results  to  documented 
Standardized  UXO 
Technology 
Demonstration  Site 
performance 

Inconsistent 

No 

Data  Density 

>  15  pts  /  m2 

Calculated  from  survey 
results 

Average  22  pts  /  m2 
yes 

These  are  discussed  fully  in  section  4  below. 

3.2  Selecting  Test  Site 

The  Former  Kirtland  Precision  Bombing  Range  (PBR)  was  selected  by  the  ESTCP  Program 
Office  due  to  a  combination  of  size,  topography,  and  development  pressure.  It  was  a  nearly  ideal 
site  for  use  of  a  ground-based  system  on  a  Wide  Area  Assessment  Pilot  Program,  as  the 
topography  was  extremely  hospitable  to  ground-based  vehicular  technology. 

3.3  Test  Site  History  /  Characteristics 

The  former  Kirtland  PBR  consists  of  a  total  of  -38,000  acres,  encompassing  multiple  target 
areas.  It  is  located  west  of  Albuquerque,  NM,  and  served  as  a  training  area  for  Kirtland  Air  Force 
Base  during  WWII.  The  WAA  Pilot  Program  was  conducted  on  two  parcels  totaling  5000  acres 
on  either  side  of  Double  Eagle  Airport.  The  study  area  is  known  to  contain  three  precision 
bombing  ranges  and  an  additional  simulated  oil  refinery  target.  Munitions  known  or  suspected  to 
have  been  used  on  the  site  include  100-lb  practice  bombs  and  250-lb  high  explosive  bombs.  A 
certificate  of  clearance  was  issued  for  one  portion  of  the  site,  Target  N3.  It  is  reported  that 
17,000  lbs  of  scrap  were  stored  in  this  area. 
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3.4  Present  Operations 

Currently  the  WAA  study  area  is  undeveloped.  Portions  are  planned  for  commercial  or  industrial 
development  within  the  next  decade,  and  airport  expansion  into  these  lands  is  possible. 

3.5  Pre-Demonstration  Testing  and  Analysis 

VSEMS  was  successfully  demonstrated  at  the  Standardized  UXO  Demonstration  Test  Site  at 
Aberdeen  Proving  Grounds  (APG)  in  2002  as  part  of  ESTCP  project  MM-0208.  In  2003, 
VSEMS  was  successfully  employed  to  survey  100  acres  surrounding  the  Jeep  Demo  Range  at  the 
Former  Lowry  Bombing  and  Gunnery  Range  in  Aurora  CO.  The  original  EM  configuration  of 
three  !4  x  14  meter  coils  was  replaced  with  five  1  x  14  meter  coils,  and  in  2004  the  system  was 
used  to  characterize  a  100  acre  site  suspected  of  containing  a  munitions-filled  trench  at  a  site 
near  Portland  International  Airport,  OR.  The  original  single  time  gate  EM61  electronics  were 
replaced  with  modem  EM61  Mk2  electronics,  and  the  system  was  used  for  five  separate 
commercial  UXO  surveys  at  formerly  used  defense  sites  at  Myrtle  Beach,  SC. 

3.6  Testing  and  Evaluation  Plan 

3.6.1  Demonstration  Set-Up  and  Start-Up 

The  equipment  was  mobilized  via  tractor  trailer  from  Newton,  MA  to  Albuquerque,  NM.  Upon 
arriving  at  the  site,  a  lockable  CONEX  box  was  procured,  large  enough  to  allow  VSEMS  to  be 
driven  inside  without  disassembling  the  system.  Pre-arranged  office  space  at  the  Double  Eagle 
Airport  was  inhabited.  GPS  base  station  monuments  were  located,  as  was  the  test  strip  that  had 
been  emplaced  by  the  ESTCP  Program  Office  and  representatives  from  CEHNC.  By  the  end  of 
the  first  day,  all  equipment  had  been  set  up  and  checked  out,  and  the  calibration  strip  had  been 
surveyed.  The  daily  maintenance  schedule  included  vehicle  oil  check,  cleaning  air  filters,  and 
greasing  the  axles  of  the  tow  platform.  Major  unforeseen  problems  included  persistent  jamming 
of  our  GPS  base-to-radio  link  equipment  by  other  GPS  equipment  used  by  the  myriad  of 
construction  teams  in  the  booming  valley  below;  the  high-mesa  survey  location  exacerbated  this 
problem  by  almost  ensuring  that  any  other  competing  signal  on  the  same  channel  would 
propagate  up  the  valley  to  our  location.  This  problem  was  ameliorated  by  setting  up  the  GPS  in 
the  northwest  comer  of  the  site  (the  highest  available),  using  a  30’  high  mast  for  the  radio 
antenna,  and  programming  our  rover  to  listen  only  to  the  channel  our  base  was  broadcasting  on, 
but  others  were  frequently  broadcasting  on  this  channel  as  well.  At  times,  the  channel  needed  to 
be  changed  several  times  a  day.  Other  unforeseen  problems  including  a  faulty  battery  charger 
boiling  the  lead/acid  vehicle  batteries,  causing  them  to  leak  acid  and  out-gas  hydrogen  sulfide 
(necessitating  immediate  replacement),  and  failure  of  tow  platform  glue  and  rivet  joints  on  the 
original  proof-of-concept  platform,  necessitating  daily  checks  and  replacement  of  broken  rivets 
with  brass  bolts. 

3.6.2  Period  of  Operation 


Survey  Begin 

21-Sep 

Survey  Break 

20-0  ct 

Survey  Return 

7-Nov 

Survey  End 

19-Nov 
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3.6.3  Area  Characterized 

The  survey  area  at  the  Kirtland  PBR  had  a  northern  and  a  southern  section.  Transects  were 
designed  to  cover  the  suspected  N2,  N3,  New  Demolition,  and  Simulated  Oil  Refinery  Target 
(SORT)  bombing  targets  in  the  northern  section.  Additional  transects  were  designed  to  cover  the 
southern  section  where  there  was  no  suspected  bombing  target.  A  total  of  192  acres  of  transects 
were  acquired.  These  transects  are  shown  in  the  figure  below. 
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Figure  3:  Transect  coverage  of  the  Kirtland  PBR 

The  100%  geophysical  survey  areas  were  selected  to  be  near  suspected  bombing  targets  as  these 
data  became  known  from  other  WAA  layers.  The  relationship  of  these  areas  to  the  suspected 
bombing  targets  can  be  seen  from  the  table  below.  These  areas  are  overlaid  on  the  Kirtland  PBR 
site  map  in  the  figure  below.  In  total,  158  acres  of  100%  geophysical  survey  areas  were 
surveyed,  broken  down  by  area  in  the  table  below. 
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UTM  Northing  (m) 

3,890,000  3,892,000  3,894,000 


Table  3:  Breakdown  of  acreage  by  100%  geophysical  survey  area 


acres 


area 

near 

surveyed 

Area  la 

N2 

7.2 

Areal b 

N2 

15.1 

Area 1c 

N2 

7.2 

Area2a 

Oil  Refinery  Target 

13.4 

Area2b 

Oil  Refinery  Target 

13.2 

Area2c 

Oil  Refinery  Target 

13.4 

Area3a 

Oil  Refinery  Target 

15.8 

Area3b 

New  Demolition 

6.2 

Area4a 

N3 

5.5 

Area4b 

N3 

8 

Area4c 

Oil  Refinery  Target 

14 

Area4d 

New  Demolition 

3 

Area4e 

South  Site 

20.3 

Area4f 

South  Site 

15.7 

- 1 - - 1 - 1 - 

330,000  332,000  334,000  336,000 

UTM  Easting  (m) 

Figure  4:  100%  geophysical  surveys  at  the  Kirtland  PBR 
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3,890,000  3,892,000  3,894,000 


3.6.4  Residuals  Handling 

Not  applicable. 

3.6.5  Operating  Parameters  for  the  Technology 

3.6.5.1  Magnetometers 

VSEMS  collects  five  channels  of  total  field  magnetometer  data,  triggered  primarily  by  the  1  PPS 
signal  from  the  GPS,  and  secondarily  by  the  PSYNC  signal  of  the  EM61  Mk2,  for  a  short  5  ms 
window  after  the  secondary  field  has  rung  down  and  before  the  next  PSYNC  pulse.  For  an 
unmodified  EM61,  PSYNC  is  a  75  Hz  signal,  hence  the  5  ms  window  for  sampling  the 
magnetometer  data  is  repeated  at  75  Hz.  The  five  magnetometers  are  spaced  0.5  meters  apart  on 
the  sensor  boom  resulting  in  a  physical  magnetometer  sensor  swath  width  of  2.0  meters.  Sensor 
standoff  is  30.5  cm.  Because  of  the  primary  1  PPS  triggering,  magnetometer  data  is  always 
correctly  triggered  by  the  GPS,  and  no  GPS-to-magnetometer  latency  correction  is  required. 

3.6.5.2  EM61 

VSEMS  collects  five  channels  of  EM61  Mk2  data,  triggered  by  a  10  Hz  triggering  signal  that  is 
generated  by  using  the  1  PPS  from  the  GPS  and  dividing  it.  Five  EM61  Mk2  electronics  boxes 
are  configured  in  a  master/slave  configuration.  Five  1  x  14  meter  EM61  coils  are  employed, 
spaced  55  cm  apart,  approximately  in  line  with  the  five  magnetometers  (the  !4  meter  EM61  coil 
width  is  measured  from  the  center  of  the  2.5  cm-wide  coil;  thus  the  edge-to-edge  size  is  actually 
55  cm).  The  edge-to-edge  EM61  physical  sensor  swath  width  is  2.75  meters,  but  since  the  data 
are  considered  to  be  in  the  center  of  the  coil,  the  “electronic”  sensor  swath  width  is  2.20  meters  - 
just  slightly  wider  than  the  magnetometer  swath  width.  Sensor  standoff  is  38.1  cm  (the  same  as 
when  the  EM61  lower  coils  are  deployed  on  their  native  wheels).  Although  the  system  is  capable 
of  using  EM61  upper  coils  and  employing  the  “D”  setting  on  the  electronics,  we  typically  do  not 
deploy  the  upper  coils  and  instead  deploy  only  the  lower  coils  and  employ  the  “4”  setting  on  the 
electronics  box.  This  uses  time-gates  sampled  at  256,  406,  706,  and  1306  psec,  respectively. 

3.6.5.3  RTKGPS 

The  VSEMS  GPS  equipment  incorporates  a  Trimble  5700  base  station,  and  three  Trimble 
MS750  rovers  and  antennas,  used  in  a  differential  RTK  configuration.  One  GPS  antenna  is 
located  over  the  center  magnetometer.  The  other  two  are  located  over  the  two  outer  EM61  coils. 
The  GGK  output  string  is  used  to  capture  time,  position,  and  fix  quality  at  a  10  Hz  update  rate. 

3.6.5.4  Number  and  Orientation  of  Passes 

For  the  transect  surveys,  we  collected  data  along  whatever  transects  we  were  directed  to  by  the 
ESTCP  program  office.  For  the  1 00%-coverage  surveys,  we  collected  data  in  the  areas  we  were 
directed  to  by  the  ESTCP  program  office,  using  passes  with  a  desired  !4  meter  overlap  (2  meter 
line  spacing)  along  whatever  axis  was  longest  to  maximize  survey  efficiency. 

3.6.5.5  Data  Density 

As  stated  above,  the  cross-track  data  spacing  is  0.5  meters  for  the  magnetometer  data,  and  0.55 
meters  for  the  EM61  data.  The  down-track  spacing  is  a  function  of  sampling  rate  and  speed.  The 
EM61  outputs  at  10  Hz,  which  is  slow  compared  to  the  magnetometers’  75  Hz  output  rate.  In 
order  to  achieve  a  nominal  down-track  data  spacing  of  one  EM61  update  per  10  cm,  the  speed 
needs  to  be  no  greater  than  1  meter/second,  or  approximately  2  mph.  Because  this  was  not  a 
detection  and  remediation  survey,  we  felt  that  a  lower  bound  on  the  down-track  data  density  of 
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one  EM61  update  per  20  cm  was  more  appropriate,  yielding  a  forward  rate  of  advance  of 
between  4  and  5  mph.  Note  that,  with  a  0.5  meter  magnetometer  spacing,  down-track  spacing  of 
20  cm  translates  into  a  density  of  15  data  points  per  square  meter,  which  is  the  number  used  in 
table  1.  Since  the  magnetometers  were  updating  at  75  Hz,  they  easily  satisfied  the  20  cm  spacing 
metric;  the  down-track  magnetometer  data  spacing  at  this  rate  was  one  update  every  2.6  cm.  As 
per  discussions  with  the  Program  Office,  we  drove  the  transects  at  less  than  4  or  5  mph  to 
achieve  a  20  cm  down-track  EM61  data  spacing,  but  on  the  100%-coverage  surveys,  we  slowed 
down  and  drove  approximately  2  mph  to  achieve  a  10  cm  down- track  EM61  data  spacing.  The 
actual  metric  listed  in  the  table  is  not  a  down-track  value,  however,  but  an  area  density  of  >15 
points  per  square  meter.  The  measured  EM61  area  density  for  the  14  100%  geophysical  survey 
areas  was  over  22  points  per  square  meter. 

3.6.5.6  Labor 

For  safety  reasons,  there  is  a  de-facto  requirement  that  the  survey  vehicle  be  in  sight  of  a  field 
person  at  all  times.  Therefore,  on  a  large  survey  site  such  as  the  Kirtland  PBR,  the  VSEMS  crew 
nominally  includes  a  driver  and  two  flaggers,  one  at  the  end  of  each  survey  line.  Because  of  the 
next-day  data  analysis  turnaround  requirement,  a  data  analyst  was  in  the  field,  analyzing  the 
previous  day’s  data  (normally  the  analyst  would  be  one  of  the  flaggers).  This  resulted  in  a  total 
crew  size  of  four. 

Site  maps  of  transect  data  and  100%  geophysical  survey  data  are  shown  in  section  4  below. 

3.6.5.7  Data  Processing  Steps 

For  the  transect  surveys,  all  data  processing  and  analysis  occurred  in  our  own  software.  For  the 
100%  geophysical  survey  surveys,  additional  processing  occurred  in  Geosoft  Oasis  Montaj™. 

Positioning:  GPS  data  are  viewed  and  corrected  as  necessary  to  fix  jumps  that  come  from  non- 
RTK  solutions.  At  the  Kirtland  PBR  site,  there  were  many  sources  of  radio  frequency  (RF) 
interference  almost  certainly  due  to  surveying  and  construction  crews  using  similar  GPS 
equipment.  As  such,  many  data  sets  required  manual  correction  of  non-RTK  solutions.  These 
corrections  take  the  form  of  “connecting  the  dots”  between  valid  RTK  solutions.  Because  the 
vehicle  was  slow-moving  and  the  transects  were  extremely  regular,  it  is  not  felt  that  correction  of 
these  outages  compromised  the  data. 

Before  this  survey,  VSEMS  had  a  single  GPS  with  its  antenna  over  the  center  magnetometer.  In 
support  of  an  EQT-funded  project,  the  system  is  being  improved  to  increase  the  accuracy  of  the 
positioning  of  the  EM61  coils.  Shortly  before  the  Kirtland  deployment,  VSEMS  was  modified  to 
include  two  additional  GPS  antennas  over  the  outboard  two  EM61  coils  and  two  additional 
Trimble  GPS  RTK  receivers.  The  10  Hz  data  stream  from  all  three  antennas  were  recorded  on  all 
transect  and  100%  geophysical  surveys.  While  in  the  field,  software  was  written  to  view  and 
position-correct  the  three  streams  of  position  data. 

The  addition  of  the  two  GPS  antennas  and  receivers  had  the  unintended  benefit  of  supplying 
redundant  equipment.  There  were  times  when  GPS1,  the  one  over  the  center  magnetometer  and 
the  one  used  by  default,  received  non-RTK  data  over  an  entire  line  or  transect,  making  that  data 
unusable  for  analysis.  However,  these  positions  could  be  recovered  because,  at  these  times, 
GPS2  and  GPS3  were  outputting  valid  data,  and  their  locations  could  be  used  to  reconstruct  the 
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location  of  GPS1.  Without  these  redundant  units,  more  lines  would  have  needed  to  be 
resurveyed. 

Magnetometer  Data:  The  magnetometer  data  are  positioned  using  the  GPS.  They  are  then 
lightly  median- filtered  to  remove  spikes,  notch-filtered  to  remove  the  15  Hz  sine  wave  that 
comes  from  the  60  Hz  ambient  electrical  hum  from  the  power  grid  that,  due  to  the  system’s  75 
Hz  sampling,  aliases  at  15  Hz.  The  data  are  then  passed  to  a  dynamic  background  filter  that 
determines  the  median  of  a  six-second  window,  then  subtracts  that  median  value  from  the  data. 
This  effectively  removes  large-scale  geology,  the  remnant  signature  of  the  vehicle,  and  diurnal 
drift  from  the  data.  The  data  are  then  gridded  on  a  10  cm  grid  and  visualized. 

EM61  Data:  The  EM61  data,  as  described  above,  are  positioned  using  the  GPS  over  the  center 
magnetometer.  Successive  GPS  updates  are  smoothed  and  used  to  determine  heading.  The 
heading,  platform  geometry,  and  GPS  position  are  used  to  determine  the  positions  of  the  centers 
of  all  five  EM61  lxl/2  meter  coils.  The  EM61s  are  powered  by  two  large  car  batteries,  and  are 
zeroed  in  the  field  in  the  morning  and  after  lunch.  This  combination  results  in  data  with  a 
minimum  of  drift,  and  as  such,  gross  background  leveling  of  the  EM61  data  was  not  necessary 
for  visual  anomaly  detection  on  the  transect  surveys.  Like  the  magnetometer  data,  the  EM61  data 
are  gridded  on  a  10  cm  grid  and  visualized.  For  100%  geophysical  surveys,  the  EM61  data  was 
background-leveled  in  Geosoft  Montaj™  using  a  nonlinear  filter. 

The  gridded  magnetometer  and  EM61  data  were  then  viewed  together  in  our  own  software  for 
anomaly  picking  (described  below). 

In  addition  to  processing  in  our  own  software.  Magnetometer  and  EM61  data  sets  were  brought 
into  Geosoft  Montaj™.  Project  files,  database  files,  grid  files,  map  files,  and  geotiff  files  were 
created  and  delivered  to  the  Program  Office. 

3.6.6  Demobilization 

The  GPS  base  station  and  its  30’  radio  mast  were  taken  down.  The  tractor/trailer  was  packed  and 
driven  off  site.  The  company  from  whom  the  CONEX  box  was  rented  was  contacted  for  its 
removal. 

3.7  Selection  of  Analytical/Testing  Methods 

Not  applicable. 

3.8  Selection  of  Analytical/Testing  Laboratory 

Not  applicable. 

4  Performance  Assessment 

4.1  Performance  Criteria 

The  following  were  the  primary  performance  objectives  from  the  demonstration  test  plan,  (these 
were  also  listed  in  section  3.1): 
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Table  4:  Primary  Transect  Performance  Objectives/Metrics  and  Confirmation  Methods  Relating  to  Detection 

of  Target  Areas  and  Target-Free  Areas 


Type  of 
Performance 
Objective 

Performance 

Criteria 

Expected 

Performance 

(Metric) 

Performance 
Confirmation  Method 

Actual  Performance 
Objective  Met? 

Qualitative 

Reliability 

and 

Robustness 

General 

Observations 

Operator  feedback  and 
recording  of  system 
downtime  (length  and 
cause) 

System  was  Reliable 

Yes 

Terrain  / 

Vegetation 

Restrictions 

General 

Observations 

Correlation  of  areas  not 
surveyed  to  available 
data  (topographical 
maps ,  etc.) 

System  surveyed 
vehicularly  navigable 
areas 

Yes 

Quantitative 

Survey  Rate 

12.5  acres  /  day 

Calculated  from  survey 
results 

9.5  acres /day 

No 

Data 

throughput 

All  data  from  day 
x  processed  for 
anomalies  and 
submitted  by  end 
of  day  x+1 

Analysis  of  records  kept 
/  log  files  generated 
while  in  the  field 

Analyzed  Anomalies 
Submitted  by  Next  Day 
Yes 

Percentage 
of  Assigned 
Coverage 
Completed 

>95%  as  allowed 
by  topography 

Calculated  from  survey 
results 

All  Transects  Completed 
Yes 

75%  of  100% 
geophysical  survey 
Areas  Completed 

No 

Transect 

Location 

95%  within  2 
meters  of 
requested 
transects 

Calculated  from  survey 
results 

93%o  of  Transects 
Aligned(after  learning 
curve) 

Yes 

The  following  were  the  secondary  performance  objectives  from  the  demonstration  test  plan 
(these  were  also  listed  in  section  3.1): 
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Table  5:  Secondary  Transect  Performance  Objectives/Metrics  and  Confirmation  Methods  Relating  to 

Characterization  of  Target  Areas 


Type  of 
Performance 
Objective 

Performance 

Criteria 

Expected 

Performance 

(Metric) 

Performance 
Confirmation  Method 

Actual  Performance 
Objective  Met? 

Qualitative 

Ability  of 
Analyst  to 
Visualize 
Targets  from 
Survey  Data 

All  targets  in 
survey  area 
identified 

Data  Analyst  feedback 
and  comparison  to  100% 
geophysical  survey  data 
/  other  demonstrators 
results 

Targets  Readily 
Visualizeable 

Yes 

Quantitative 

Location  of 

Inverted 

Anomalies 

<  0.15  m 
horizontal 
<30%  vertical 

Comparison  to  Test  Strip 
Ground  Truth 

.13  m  horizontal 
.20  m  vertical 

Yes 

Probability 
of  False 

Alarm 

<5%  of  identified 
anomalies 
correspond  to  no 
ferrous  metal 
source 

Validation  Sampling 
(100%  survey)  and/or 
Remediation  Sampling 
(digging) 

13.5%  correspond  to 
apparent  geology 

No 

Signal  to 

Noise  Ratio 
(SNR)  for 
Calibration 
Objects 

+/- 10%  of 
expected  from 
Standardized 
UXO  Technology 
Demonstration 
Site  Performance 

Comparison  of 
Calibration  Target 
results  to  documented 
Standardized  UXO 
Technology 
Demonstration  Site 
performance 

Inconsistent 

No 

Data  Density 

>  15  pts  /  m2 

Calculated  from  survey 
results 

>  22  pts  /  m2 

Yes 

4.2  Performance  Confirmation  Methods 

The  above  two  tables  are  discussed  below. 

4.2.1  Primary  Metrics 

Reliability  and  Robustness:  The  system’s  reliability  and  robustness  were  generally  good  for  a 
prototype  system.  During  the  eight-week  survey,  two  full  days  were  lost  to  downtime.  One  was 
due  to  the  battery  charger  malfunctioning  and  boiling  the  acid  in  the  vehicle  batteries,  and 
necessitating  their  immediate  replacement.  The  other  was  due  to  chasing  down  and  solving  a 
noise  problem  on  the  magnetometers  due  to  a  current  ground  loop.  The  proof-of-concept 
fiberglass  towed  platform  degraded  during  the  survey,  as  its  glued-and-riveted  joints  began  to 
fail.  Near  the  end  of  one  survey  day,  one  assembly  holding  the  EM61  array  failed  completely.  It 
was  repaired  that  evening  with  brass  bolts.  On  all  subsequent  days,  we  inspected  the  platform 
every  morning  and  replaced  any  missing  rivets  with  brass  bolts,  with  no  further  downtime  due  to 
platform  malfunction. 

Terrain/Vegetation  Restrictions:  There  were  very  few  terrain  or  vegetation  restrictions;  the 
former  Kirtland  PBR  was  generally  a  flat,  open  area  with  low,  scrubby  vegetation.  The  main 
restrictions  came  from  several  fence  lines  cutting  across  the  site,  preventing  some  of  the  transects 
from  being  followed  exactly  as  planned. 
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Survey  Rate:  The  average  survey  rate  was  9.5  acres,  averaged  over  all  survey  days,  including 
days  when  GPS  interference  severely  hobbled  productivity,  down  days,  and  days  when  we 
switched  from  transects  to  100%  geophysical  surveys.  As  stated  above,  the  persistent  jamming  of 
the  GPS  base-to-rover  radio  link  was  the  single  largest  factor  affecting  productivity.  Despite 
locating  the  GPS  base  station  in  the  northwest  comer  of  the  site  (the  highest  point),  using  a 
scanner  to  find  an  open  channel,  and  putting  the  base  station  radio  antenna  on  a  30’  high  mast,  on 
most  days  the  base/rover  link  would  be  severed  when  another  differential  GPS  system  in  the 
valley  was  switched  on;  we  could  hear  the  interference  on  the  previously  unused  channel  on  our 
scanner.  We  would  then  have  to  locate  a  new  clear  channel  with  the  scanner,  drive  up  to  the  base 
station  in  the  comer  of  the  site  and  change  the  channel,  and  do  the  same  in  the  rover.  On  some 
days  this  procedure  needed  to  be  repeated  two  or  three  times.  On  the  days  when  there  was  no 
such  interference,  the  system  and  personnel  were  certainly  capable  of  high  productivity;  nine 
days  were  over  12.5  acres  per  day,  six  were  over  15,  and  one  was  over  20. 

Data  Throughput:  Data  was  analyzed  and  anomaly  lists  were  supplied  to  the  ESTCP  Program 
Office  by  the  end  of  the  next  day. 

Percentage  of  Assigned  Completed  Coverage:  All  of  the  planned  transects  were  covered  (with 
the  exception  of  N/S  transects  though  the  northwestern  comer  of  the  site  that  were  deleted  in 
concurrence  with  the  program  office  because  the  target  they  were  designed  to  find  had  been 
found).  All  of  the  100%  geophysical  survey  areas  were  surveyed,  but  in  acreage,  75%  of  the 
assigned  total  coverage  was  accomplished.  This  is  because  the  first  three  assigned  100% 
geophysical  survey  areas  were  30  acres  each,  and  we  were  initially  instructed  to  survey  as  much 
of  each  one  as  could  be  completed  in  a  day.  All  subsequent  100%  geophysical  survey  areas  were 
100%  surveyed. 

Transect  Location:  Transect  locations  from  the  first  two  days  were  off  by  as  much  as  tens  of 
meters  due  to  learning  curve  issues  in  programming  and  following  the  track  guidance  equipment 
in  the  vehicle,  but  subsequently,  the  geodetically-located  sensor  swath  locations  overlaid  the 
planned  transects  in  nearly  all  data  sets.  A  numerical  off-track  metric  was  obtained  by 
calculating,  for  every  transect,  the  average  orthogonal  distance  from  the  nearest  planned  transect. 
These  results  are  shown  in  the  figure  below.  The  degree  to  which  track  following  improved 
enormously  after  the  second  day  is  clear.  The  magenta  line  represents  the  2-meter  metric.  When 
including  the  poor  results  from  the  first  two  days,  the  percentage  of  tracks  within  2  meters  of  the 
planned  tracks  is  93%,  just  missing  the  95%  metric,  but  if  these  learning-curve  days  are  excluded 
from  the  calculation,  98%  of  the  tracks  are  within  the  metric.  81%  of  the  tracks  were  within  1 
meter  of  the  planned  transects,  and  53%  were  with  14  meter. 
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Distance  Off-Track  From  Planned  Traverses 
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Figure  5:  Distance  Off-Track  from  Planned  Traverses.  The  2-meter  metric  is  represented  by  the  magenta 

line. 


4.2.2  Secondary  Metrics 

Ability  to  Visualize  Targets:  Targets  (both  individual  anomalies  and  entire  bombing  targets) 
were  readily  visualizable  in  VSEMS  survey  data. 

Location  of  Inverted  Anomalies:  Unfortunately,  no  independent  measurement  was  made  of  the 
actual  location  of  the  ground  truth  items  that  were  dug.  In  the  absence  of  this  information,  we 
analyzed  the  location  accuracy  of  the  test  strip  items,  and  found  average  accuracies  of  13  cm 
horizontally  with  a  standard  deviation  of  5.2  cm,  and  20  cm  vertically  with  a  standard  deviation 
of  7.7  cm. 

Probability  of  False  Alarm:  We  will  address  the  false  alarm  issue  in  detail,  in  section  4  below, 
as  it  goes  to  the  heart  of  the  utility  of  a  concurrent  multisensor  system.  A  summary  of  the 
probability  of  false  alarm  is  below. 
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Table  6:  Reduction  in  False  Alarms  with  Reanalysis  of  Data 


Probability  of  false  alarm  (no-finds  probable  geology)  from  original  Program 

Office  Dig  Sheet  (includes  anomalies  from  VSEMS  ground-based  data  plus  other 
data) 

18% 

Probability  of  false  alarm  using  anomalies  only  from  VSEMS  ground-based  data 
on  the  14  100%  geophysical  survey  areas  for  which  there’s  ground  truth  on  the 
Program  Office  Dig  Sheet 

13.5% 

Probability  of  false  alarm  removing  objects  whose  mag  signatures  are  clearly  non- 
dipolar  (the  best  the  analyst  could  do  with  mag-only  data) 

8% 

Probability  of  false  alarm  requiring  a  confirming  signature  in  the  EM61  data 

1% 

Signal  to  Noise  for  Calibration  Objects:  The  signal  from  calibration  objects  was  not 
sufficiently  consistent  to  meet  the  desired  metric,  but  this  is  more  a  reflection  of  platform  motion 
than  it  is  a  symptom  of  anything  wrong;  this  is  discussed  in  detail  in  section  4  below. 

Data  Density:  The  data  density  in  the  100%  geophysical  survey  areas,  from  the  magnetometers 
was,  on  average,  165  points  per  square  meter.  The  data  density  from  the  EM61s  was,  on  average, 
22  points  per  square  meter. 

4.3  Data  Analysis,  Interpretation  and  Evaluation 
4.3.1  Test  Strip  Data 

The  test  strip  was  emplaced  by  Program  Office  representatives  and  contained  the  15  items  listed 
in  the  table  below. 


Table  7:  Test  strip  items 


ID 

Item 

Easting 

(m) 

Northing 

(m) 

Elevation 

Description 

comment 

civil 

survey 

depth 

(cm) 

measured 

depth 

(cm) 

100 

7 

335959.849 

3892136.536 

1773.654 

#4  RBR-  100m 

rebar 

200 

15 

336059.693 

3892134.106 

1773.108 

#4  RBR  -  200m 

rebar 

1000 

1 

335859.939 

3892139.374 

1774.14 

#4  RBR  -  0m  10 

rebar 

10  R 

3 

335870.63 

3892139.87 

1773.86 

10  ROUND 

shotput  - 16  lb. 

21.3 

25.40 

107 M 

8 

335967.129 

3892136.729 

1773.184 

107  MIDDLE 

81  mm  mortar 

36.1 

40.64 

1 13 M 

9 

335972.98 

3892136.491 

1773.468 

113  MIDDLE 

60  mm  mortar 

6.7 

10.16 

128M 

10 

335987.547 

3892136.206 

1772.867 

128  MIDDLE 

105  mm  Proj. 

59.6 

60.96 

144R 

11 

336003.792 

3892135.719 

1773.365 

144  ROUND 

shotput  - 16  lb. 

7.2 

10.16 

165JVI 

12 

336024.826 

3892135.665 

1772.915 

165  MIDDLE 

155  mm  Proj. 

4.7 

48.26 

180 M 

13 

336039.83 

3892134.869 

1772.443 

180  MIDDLE  180 

155  mm  Proj. 

79.9 

91.44 

195 M 

14 

336054.724 

3892134.429 

1772.877 

195  MIDDLE 

60  mm  mortar 

22.5 

25.40 

23 M 

4 

335883.353 

3892139.827 

1774.069 

23  MIDDLE 

37  mm  Proj. 
simulant 

-0.5 

5.08 

39 M 

5 

335899.42 

3892139.428 

1773.627 

39  MIDDLE 

105  mm  Proj. 

37.4 

40.64 

4 M 

2 

335864.045 

3892139.522 

1774.072 

4  MIDDLE 

37  mm  Proj. 
simulant 

9.1 

25.40 

66_M 

6 

335926.44 

3892138.068 

1773.529 

66  MIDDLE 

81  mm  mortar 

26.9 

26.67 
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The  yellow  highlighting  comes  directly  from  the  file  “WAA_kirtland_cal_strip_vl.xls” 
distributed  by  the  Program  Office.  A  comment  embedded  in  that  file  states  that  the  highlighting 
“means  that  the  civil  measurements  don’t  agree  well  with  the  in-field  measurements.” 


Figure  6:  Sample  Test  Strip  Data  with  Ground  Truth  Overlaid,  Mag 
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Figure  7:  Sample  Test  Strip  Data  with  Ground  Truth  Overlaid,  EM61  Gate3 

The  test  strip  was  used  to  determine  detection  thresholds  to  employ  on  the  site.  In  the  figure 
below,  magnetometer  data  from  the  test  strip  are  shown  and  color  highlighting  above  a  given 
threshold  is  employed.  We  do  this  in  our  own  software  as  the  color  thresholding  in  Oasis  is  a  bit 
cumbersome.  The  top  panel  shows  magnetometer  data  displayed  at  +/-50nT  in  grayscale  with  no 
color  highlighting.  The  37mm  stimulant  has  virtually  no  signature,  but  all  other  objects  do.  The 
middle  panel  shows  the  data  highlighted,  with  everything  over  lOnT  colored  in  red.  The  lOnT 
threshold  detects  all  items  except  the  37mm  stimulant,  with  one  false  alarm,  probably  due  to 
geology.  In  the  third  panel,  the  threshold  is  dropped  to  5nT,  which  produces  ten  false  alarms, 
probably  due  to  geology.  This  supported  the  use  of  a  lOnT  magnetometer  detection  threshold.  (A 
lOnT  detection  threshold  is  also  three  times  a  nominal  system  noise  floor  of  3nT.) 
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Figure  8:  Magnetometer  test  strip  data  (200  meters)  showing  that  lowering  detection  threshold  brings  in 

likely  geology 


For  EM61  data  analysis,  gate3  was  used,  as  it  frequently  is  for  production  geophysical  surveys  of 
bombing  ranges  suspected  of  containing  medium  to  large  objects.  The  figure  below  shows  EM61 
gate3  data  over  the  test  strip.  The  top  panel  is  un-highlighted;  the  bottom  panel  is  highlighted 
with  all  data  >3  mV  colorized  in  red.  All  objects  test  strip  objects  were  detected  in  the  EM61 
data  at  this  threshold.  However,  the  maximum  depth  of  objects  in  the  test  strip  was  <  1  meter,  so 
there  was  the  possibility  that  the  magnetometers  would  still  be  needed  to  detect  objects  at  greater 
depths  that  the  EM61  could  not  detect.  In  retrospect,  the  3mV  EM  detection  threshold  proved  to 
be  a  bit  low,  resulting  in  a  large  number  of  probable  clutter  picks  that  the  operator  assigned  as 
low-likelihood  anomalies. 
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155mm/ 


Test  Strip  EM61  Gate3  Data  +-  SO  mV,  >  3  mV  Hightighted  in  Red 


Figure  9:  EM61  gate3  test  strip  data  (200  meters)  with  anomalies  >3  mV  highlighted  in  red 


The  test  strip  was  run  over  in  the  morning  and  again  at  the  end  of  the  day  to  verify  correct  sensor 
operation.  Test  strip  data  was  analyzed  by  extracting  and  plotting  the  peak  sensor  value,  as  well 
as  the  peak  fit  magnetometer  value  over  each  item.  The  fit  data  was  employed  in  the  hope  that 
they  would  be  more  consistent  than  the  raw  data.  These  data  were  plotted  and  presented  to 
ESTCP  at  the  Wide  Area  Assessment  Data  Exchange  Meeting  in  November  2005.  The  data  were 
summarized  in  the  WAA  Pilot  Program  Data  Report  for  UX-0531  submitted  to  ESTCP  on 
12/29/2005.  We  will  display  three  of  the  plots  here. 

The  strongest  item  in  the  plot  was  item  12,  the  155mm  shell  at  90cm  depth.  The  run-to-run  plots 
are  displayed  in  the  figure  below.  The  blue  plot  is  raw  peak  magnetometer  data  in  a  pre-set  area 
of  interest  over  the  object,  the  magenta  plot  is  the  peak  value  of  the  inverse-modeled  (fit)  data 
over  the  object,  and  the  yellow  plot  is  the  raw  peak  EM61  gate3  value.  On  this  plot,  the  raw 
magnetometer  data  range  from  449  to  557  nT,  which  is  close  to  being  consistent  to  within  +/- 
10%.  However,  the  plots  for  other  weaker  objects  show  data  that  are  less  consistent. 


31 


Test  Strip,  Item  12  (155mm  at  90cm) 


Figure  10:  Test  strip  data,  raw  EM  value  (yellow)  and  raw  and  fitted  mag  values  (blue  and  magenta)  for 

155mm  projectile  at  90cm  depth 

Item  5,  which  was  the  105mm  projectile  at  40cm,  is  shown  below.  The  EM61  gate3  data  vary 
from  41  to  64m V;  the  magnetometer  data  vary  from  60  to  85nT.  We  feel  that  the  lack  of 
consistency  was  largely  due  to  slight  differences  in  towed  platform  position  on  the  test  strip,  and 
to  platform  motion  as  it  went  down  the  test  strip.  It  is  noted  that  there  is  a  slight  upward  drift  in 
these  data.  It  is  possible  that  this  is  due  to  both  the  rutting  of  the  test  track  with  increasing  use, 
and  to  a  slight  sagging  of  the  towed  platform  that  was  observed  over  the  duration  of  the  eight- 
week  survey.  The  presence  of  some  visual  correlation  between  the  dips  in  the  EM61  and 
magnetometer  plots  lend  credence  to  the  suggestion  that  the  lack  of  run-to-run  consistency  is  due 
to  platform  motion.  The  peak  values  of  the  fit  magnetometer  data  show  the  same  lack  of 
consistency,  indicating  that  the  raw  peak  values  are  not  simply  spuriously  high  readings  -  the 
region  of  data  in  the  area  immediately  surrounding  the  object  is  biased  up  or  down.  This  is  why, 
even  after  fitting  with  a  point  dipole,  the  peak  value  of  the  fit  data  is  largely  tracking  the  real 
data.  Note  that  the  median  filter  that  is  employed  to  remove  long-wavelength  effects  such  as 
broad  scale  geology  does  not  compensate  for  this  because,  by  design,  the  median  filter  operates 
with  a  window  sufficiently  long  (six  seconds)  that  it  has  little  effect  on  spatial  wavelengths  the 
size  of  an  MEC  object. 
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Test  Strip,  Item  5  (105mm  at  40cm) 
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EM  Raw 


Figure  11:  Test  strip  data,  raw  EM  value  (yellow)  and  raw  and  fitted  mag  values  (blue  and  magenta)  for 

105mm  projectile  at  40cm  depth 

For  comparison,  data  from  one  of  the  weakest  items  is  plotted  below  -  the  37mm  stimulant  at 
25cm  depth.  This  object  was  barely  visible  to  the  magnetometers  at  all,  but  had  a  clear  signature 
to  the  EM61.  The  magnetometer  data  range  from  7  to  27  nT;  the  EM61  gate3  data  from  11  to 
23mV.  This  numbers  clearly  are  not  within  the  desired  +/- 1 0%  of  accepted  readings  from  the 
Standardized  UXO  Demonstration  Test  Sites.  The  likely  cause  is  a  combination  of  the  towed 
fiberglass  platform  used  for  the  Kirtland  PBR  survey,  the  motion  engendered  in  the  platform  by 
uneven  terrain,  and  variation  in  vehicle  path.  This  platform  was  originally  developed  under 
ESTCP  project  MM-0208  to  host  three  14  x  14  meter  coils  and  was  only  designed  to  survive  a 
single  APG  deployment.  Prior  to  the  Kirtland  PBR  survey,  the  platform  was  modified  and 
reinforced  to  host  five  1  x  14  meter  coils  -  nearly  four  times  the  weight  of  the  original  coils.  To 
maximize  the  survivability  of  the  platform,  it  employed  a  “porch  swing”  design  that  allowed  the 
coils  to  swing  back  upon  encountering  a  low  obstacle  such  as  a  tree  trunk.  This  degree  of 
freedom  allowed  the  EM  coils  to  sway  slightly  even  without  encountering  obstacles,  and  added 
to  the  uncertainty  of  the  position  of  the  coils  relative  to  the  platform.  The  fiberglass  platform  has 
since  been  replaced  with  a  carbon  fiber  platform  built  with  the  specific  design  goal  of  having  as 
little  unregistered  motion  of  the  EM61  coils  as  possible;  rigidly-mounted  EM61  coils  are  now 
employed.  The  magnetometers  on  the  fiberglass  platform  show  similar  variations  in  signal.  Our 
experience  is  that  it  doesn’t  take  much  terrain-induced  variation  for  the  1/R3  (mag)  or  1/R6  (EM) 
signal  dependency  to  produce  the  variations  seen  on  the  test  strip. 
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Test  Strip,  Item  2  (37mm  simulant  at  25cm) 
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Figure  12:  Test  strip  data,  raw  EM  value  (yellow)  and  raw  and  fitted  mag  values  (blue  and  magenta)  for 

37mm  simulant  at  25cm  depth 


We  have  generated  these  plots  for  all  15  test  strip  items  and  have  included  them  in  an  appendix. 

4.3.2  Transect  Data 

Because  of  the  large  amount  of  transect  data,  only  a  subset  will  be  shown.  For  most  areas,  we 
were  given  “sparse”  transects  spaced  150  meters  apart  and  “dense”  transects  spaced  75  meters 
apart  to  run,  typically  in  orthogonal  directions.  With  these  transects,  the  VSEMS  ground-based 
data  easily  located  the  known  targets  N2  and  N3,  and  helped  to  locate  the  Simulated  Oil  Refinery 
Target  (SORT).  The  figure  below  shows  the  site  with  the  suspected  target  areas  and  subsequently 
derived  areas  of  interest  overlaid,  with  a  red  circle  shown  over  the  N2  and  New  Demolitions 
targets.  The  following  two  figures  below  show  EM61  data  from  the  sparse  WE  transects  and  the 
dense  NS  transects  over  this  area,  and  magnetometer  data  from  the  same  area. 
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Figure  13:  N2  and  New  Demolitions  Target 
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Figure  15:  Sparse  WE  transects  (150m  spacing)  and  dense  NS  transects  (75m  spacing)  over  N2  (circle,  left) 
and  new  demolition  (circle,  right)  targets,  mag  data  (+/-  50  nT) 
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Below,  representative  magnetometer  and  EM61  data  are  shown  from  the  SORT  target.  The 


Figure  16:  Red  circle  denotes  location  of  Simulated  Oil  Refinery  Target  (SORT) 
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Figure  17:  Sparse  WE  transects  (150m  spacing)  and  dense  NS  transects  (75m  spacing)  over  SORT,  EM61 

gate3  data  (+/-  50  mV) 
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Figure  18:  Sparse  WE  transects  (150m  spacing)  and  dense  NS  transects  (75m  spacing)  over  SORT,  mag  data 

(+/-  50  nT) 
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The  red  circle  in  the  figure  below  shows  the  location  of  the  N3  target.  The  next  two  images  show 
the  sparse  and  dense  traverses  over  that  target. 


Figure  19:  N3  target 
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Figure  20:  Sparse  WE  transects  (150m  spacing)  and  dense  NS  (50m  spacing)  over  target  N3  (red  circle), 

EM61  gate3  data  (+/-  50  mV) 
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Figure  21:  Sparse  WE  transects  (150m  spacing)  and  dense  NS  (50m  spacing)  over  target  N3  (red  circle),  mag 

data  (+/-  50  nT) 


The  images  below  show  transect  data  acquired  in  the  southern  part  of  the  site. 
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Figure  22:  Sparse  WE  transects  (150m  spacing)  over  southern  section,  EM61  gate3  data  (+/-  50  mV) 


44 


Figure  23:  Sparse  WE  transects  (150m  spacing)  over  southern  section,  mag  data  (+/-  50  nT) 


For  transect  data,  all  analysis  was  performed  using  our  own  data  analysis  software.  All  target 
picking  was  performed  manually.  Magnetometer  data  and  EM61  data  were  simultaneously 
displayed  on  the  screen  using  linked  copies  of  our  software.  Panning,  zooming,  and  scrolling  in 
one  set  of  data  (e.g.,  magnetometer  data)  results  in  the  same  action  in  the  other  set  (e.g.,  EM61 
gate3  data).  Typically,  because  the  magnetometer  data  contains  a  superset  of  anomalies  due  to 
the  more  sensitive  1/R3  response,  magnetometer  data  is  viewed  as  the  “primary”  sensor,  and 
areas  of  interest  are  drawn  around  anomalies  in  the  magnetometer  data.  This  automatically 
results  in  the  same  area  being  drawn  around  the  corresponding  location  in  the  EM61  gate3  data. 
The  detection  thresholds  determined  from  the  test  strip  (10  nT  for  magnetometer  data,  3  mV  for 
EM61  gate3  data)  are  used  as  a  visualization  aid,  with  any  data  above  the  specified  threshold 
colorized  in  red.  This  colorization  is  typically  flipped  on  and  off  by  the  operator  so  he  can  see 
can  see  variations  in  what  would  otherwise  be  solid  blotches  of  color. 

The  software  then  inverts  the  magnetometer  data  inside  the  specified  area  of  interest  using  a 
model  of  a  point  dipole.  Parameters  from  the  inversion  include  location,  depth,  size,  moment, 
and  goodness  of  fit  estimates;  these  are  recorded  in  a  file,  as  is  an  optional  operator-entered 
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comment  field.  The  peak  sensor  value  and  peak  fit  value  are  also  recorded.  Our  software  has  no 
EM  inversion  capability,  but  the  peak  EM  sensor  value  is  recorded  in  the  file. 

As  discussed  at  length  above,  the  operator  then  enters  a  “likelihood”  value  intended  to  broadly 
triage  candidate  anomalies  into  three  different  classes.  Likelihood  2  anomalies  are  those  that  the 
operator  feels  are  of  a  size  and  shape  consistent  with  whole  a  compact  ferrous  object.  These 
typically  have  magnetic  signatures  that  are  classic  dipoles,  with  a  clear,  round,  strong,  well- 
defined  positive  lobe,  and  a  clear,  mushroom  cap-shaped,  strong,  well-defined  negative  lobe.  The 
operator  looks  at  both  magnetometer  and  EM61  data  and  uses  his  judgment  when  making  the 
likelihood  determination.  The  anomaly  may  have  a  weak  or  no  magnetic  signature,  but  may  still 
be  flagged  as  Likelihood  2  if  the  EM61  signal  is  broad  and  strong  enough.  Likelihood  0 
anomalies  are  those  that  the  operator  feels  are  due  to  noise  or  geology  in  either  the  magnetometer 
or  EM61  data.  Likelihood  1  anomalies  are  those  that  are  neither  Likelihood  2  or  likelihood  0. 
These  are  generally  anomalies  that  have  a  discernible  signature  in  the  magnetometer  and/or 
EM61  data,  but  are  not  the  largest  strongest  anomalies  in  the  data  set,  and  are  smaller  and/or 
weaker  than  the  candidate  anomalies  in  the  test  strip.  Anomalies  with  a  spatial  extent  consistent 
with  frag  were  classified  as  Likelihood  1 . 

After  the  operator  has  viewed  all  of  the  magnetometer  data,  the  data  sets  are  “flipped,”  and  the 
EM61  data  is  viewed  as  the  primary  sensor.  With  all  of  the  anomalies  selected  thus  far  overlaid 
on  the  EM61  data,  the  operator  then  carefully  combs  the  data  and  selects  any  anomalies  above 
the  EM  threshold  that  were  not  selected  from  the  magnetometer  data  because  they  had  no 
signature  or  too  weak  a  signature.  These  are  then  added  to  the  data  set. 

The  following  table  summarizes  the  number  of  anomalies  from  each  submitted  set  of  transect 
data,  and  the  breakdown  of  anomalies  by  likelihood. 
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Table  8:  Breakdown  of  anomalies  by  day  and  likelihood 


number  of 
date  anomalies 


21 -Sep 

144 

22-Sep 

569 

23-Sep 

180 

24-Sep 

185 

26-Sep 

99 

27-Sep 

483 

28-Sep 

13 

30-Sep 

161 

1-Oct 

218 

4-Oct 

655 

5-Oct 

261 

6-Oct 

279 

1 0-Oct 

302 

1 4-Oct 

231 

1 5-Oct 

36 

18-Oct 

78 

19-Oct 

1680 

20-Oct 

210 

12-Nov 

1458 

14-Nov 

225 

total 

7467 

#  of  L  2 

3597 

#  of  L  1 

2457 

#  of  L  0 

1413 

7467 

mag  primary 

5875 

EM  primary 

1592 

It  should  be  understood  that,  because  anomalies  are  extracted  from  the  mag  and  EM61  data 
simultaneously,  it  is  misleading  to  talk  about  “mag  anomalies”  and  “EM  anomalies.”  There  are 
not  separate  lists;  there  is  a  single  list  containing  all  selected  anomalies,  and  entries  in  the  list 
include  the  peak  mag  and  EM  signals  in  the  selected  area,  as  well  as  which  sensor  was  viewed  as 
the  primary  sensor  during  analysis.  If  it  is  important  to  answer  the  question  “did  this  sensor  see 
object  X,”  one  could  look  at  which  sensor  was  the  primary  sensor,  and  combine  that  with  the 
peak  sensor  reading,  or  re-examine  the  original  data. 

4.3.3  100%  Geophysical  Survey  Area  Data 

Fourteen  100%  geophysical  survey  areas  were  specified  by  the  ESTCP  Program  Office  and 
surveyed  with  VSEMS.  The  locations  of  these  areas  relative  to  the  Kirtland  PBR  site  are 
depicted  in  figure  4  above.  The  28  individual  site  images  (14  sites,  mag  and  EM61)  are  below, 
with  brief  comments.  A  more  complete  discussion  of  the  data  and  results  follows  the  images. 
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Areal  a,  arealb,  and  areal  c  were  located  near  target  N2,  situated  radially  outward  from  the  target 
to  collect  data  from  which  anomaly  falloff  and  background  information  could  be  ascertained. 
After  several  weeks  of  performing  widely-spaced  transect  surveys,  areal  a  was  the  first  100% 
geophysical  survey  area  we  attempted.  Due  to  operator  error,  the  line  spacing  (set  by  visually 
positioning  the  vehicle’s  tire  tracks)  was  close  enough  that  only  minor  missed  area  appears  in  the 
EM61  data.  However,  more  noticeable  missed  area  appears  in  the  magnetometer  data.  This 
apparent  discrepancy  is  due  to  the  fact  that  the  five  magnetometers  are  on  50cm  centers  yielding 
a  2  meter  swath,  whereas  the  EM61s  due  to  the  physical  coil  width  are  on  55cm  centers,  yielding 
a  2.2  meter  swath  width.  After  this  data  set  was  viewed,  we  carefully  calculated  the  spacing  of 
tire  tracks  needed  on  adjacent  survey  lines  to  overlap  the  outboard  magnetometer  on  line  N  with 
the  inboard  magnetometer  on  traverse  N+l,  and  verified  that  the  lines  were  being  driven  in  this 
manner.  Area  la  contained  347  anomalies,  of  which  127  were  flagged  by  the  analyst  as 
Likelihood  2  anomalies.  Arealb  contained  209  anomalies,  of  which  49  were  flagged  by  the 
analyst  as  Likelihood  2  anomalies.  Arealc  contained  48  anomalies,  of  which  1 1  were  flagged  by 
the  analyst  as  Likelihood  2  anomalies.  Particularly  in  the  magnetometer  data  of  areal  a,  many  of 
the  Likelihood  2  anomalies  are  of  a  very  consistent  size  and  shape.  Digging  verified  that  the 
majority  of  these  were  parts  from  M38  bombs.  The  falloff  in  number  of  likelihood  2  anomalies 
per  acre  is  depicted  in  the  able  below. 


Table  9:  Falloff  in  Anomaly  Density  Near  Target  N2 


number  of 
likelihood  2 
anomalies 

near  target 

acreage 

anomaly  density 
(number  per 
acre) 

Areal  a 

127 

N2 

7.2 

17.6 

Arealb 

149 

N2 

15.1 

9.9 

Arealc 

11 

N2 

7.2 

1.5 
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Figure  24:  areal  a  near  target  N2,  7.2  acres,  EM61  gate  3,  +/-  50  mV 
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Figure  25:  areal  a  near  target  N2,  7.2  acres,  mag,  +/-  50  nT 
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Figure  26:  arealb  near  target  N2, 15.1  acres,  EM,  gate3,  +/-  50mV 
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Figure  27:  arealb  near  target  N2, 15.1  acres,  mag,  +/-  50nT 
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Figure  29:  arealc  near  target  N2,  7.2  acres,  mag,  +/-  50nT 


Area2a,  area2b,  and  area2c  were  situated  radially  outward  from  the  previously  unknown  target  in 
the  area  of  the  Simulated  Oil  Refinery  Target  (SORT).  Area2a  in  the  figures  below  show  four 
large,  round  features  the  analyst  affectionately  deemed  “crop  circles.”  These  features  are  large 
(23  meters  in  diameter),  and  lie  on  the  comers  of  a  108  meter  square.  Most  interesting,  they  ring 
out  quite  clearly  in  the  EM61  data,  but  are  very  difficult  to  see  in  the  magnetometer  data.  A  total 
of  160  anomalies  were  analyzed  in  area2a,  81  of  which  were  flagged  as  Likelihood  2.  A  total  of 
1 1 1  anomalies  were  analyzed  in  area2b,  38  of  which  were  flagged  as  Likelihood  2.  A  total  of  80 
anomalies  were  flagged  in  area2c,  15  of  which  were  flagged  as  Likelihood  2.  As  was  the  case 
with  area  la,  area2a  contains  many  anomalies,  particularly  in  the  magnetometer  data,  of  a  very 
consistent  size  and  shape,  and  digging  also  revealed  that  most  of  these  were  parts  of  M38  bombs. 
The  falloff  in  the  number  of  Likelihood  2  anomalies  per  acre  is  depicted  in  the  table  below. 
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Table  10:  Falloff  in  Anomaly  Density  Near  SORT 


number  of 
likelihood  2 
anomalies 

near  target 

acreage 

anomaly 
density 
(number 
per  acre) 

Area2a 

81 

SORT 

13.4 

6.0 

Area2b 

38 

SORT 

13.2 

2.9 

Area2c 

15 

SORT 

13.4 

1.1 
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Figure  30:  area2a  near  SORT,  13.4  acres,  EM,  gate3,  +/-  50mV 
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Figure  31:  area2a  near  SORT,  13.4  acres,  mag,  +/-  50nt 


Below  are  blowups  of  one  of  these  circular  anomalies.  In  the  first  set  of  images,  EM61  gate3 
data  are  shown  at  two  different  display  scales.  In  the  second  set,  magnetometer  data  are  shown  at 
two  different  display  scales.  The  anomalies  are  faintly  visible  in  the  magnetometer  data,  but  were 
not  apparent  to  the  analyst  until  he  saw  them  in  the  EM61  data.  It  is  unlikely  that  they  would 
have  been  seen  had  a  magnetometer-only  survey  been  performed.  The  origin  of  these  anomalies, 
and  the  physical  material  creating  them,  is  unknown;  since  it  is  in  the  area  associated  with  the 
SORT,  our  conjecture  is  that  these  are  remnants  of  large  metal-skinned  structures  such  as  tanks. 
The  centers  of  two  of  these  circular  anomalies  were  flagged  as  Likelihood  2  targets,  with  a 
comment  in  the  target  file  indicating  their  diameter,  but  they  both  were  reported  on  the  dig  sheet 
as  no-finds.  It  is  unknown  if  the  reconnaissance  included  examination  of  the  ground  not  only  at 
the  center,  but  at  the  radius  we  were  reporting.  The  clear  magnetic  dipole  circled  in  red  in  the 
magnetometer  image  below  is  object  k-61 1,  which  is  listed  in  the  dig  sheet  as  “M38  bomb  parts.” 
Since  the  location  of  this  object  is  right  on  the  circumference  of  one  of  the  circles,  and  since  no 
23-meter  aluminum  cistern  was  reported  when  it  was  dug,  it  is  possible  that  the  circular 
anomalies  are  due  to  flaked-off  or  rusted  remnants  structures  that  have  long  since  been  removed. 
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Figure  32:  EM61  gate3  data  at  +/-  50mV  (left)  and  +/-10mV  (right)  of  the  lower  right  “crop  circle.”  Circle  is 

23  meters  in  diameter. 


Figure  33:  Magnetometer  data  at  +/-  50nT  (left)  and  +/-10nT  (right)  of  the  lower  right  “crop  circle.” 
Magnetic  dipole  circled  in  red  is  object  k-611.  Only  M38  bomb  parts  were  discovered  when  this  location  was 
dug,  making  it  unlikely  that  whatever  is  causing  these  circles  are  actually  large  metallic  structures  still  inset 

into  the  ground. 


In  the  image  below,  LIDAR  data  from  URS  Corporation  is  displayed  with  VSEMS  EM61  gate3 
data  overlaid.  From  here,  it  is  seen  that  the  centers  of  the  “crop  circles”  overlay  exactly  on  the 
centers  of  the  grid  squares  that  the  LIDAR  detected.  This  reinforces  the  conjecture  that  there 
were  structures  of  some  sort  in  the  middle  of  these  grid  squares. 
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Figure  34:  EM61  Mk2  gate3  data  from  area2a  near  the  suspected  SORT  target,  overlaid  on  LIDAR  data.  The 
“crop  circles”  are  exactly  correlated  with  the  centers  of  the  grid-like  features  uncovered  by  the  LIDAR. 
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Figure  35:  area2b  near  SORT,  13.4  acres,  EM,  gate3,  +/-  50mV 

In  the  magnetometer  data  from  area2b  below,  the  density  of  anomalies  typical  of  compact 
ferrous  objects  is  relatively  low,  which  allows  the  eyeball  to  be  drawn  to  the  irregular  non- 
dipolar  features  of  the  anomalies  that  are  likely  geology.  Even  at  this  coarse  distance  scale 
showing  the  entire  13.4  acres,  it  is  clear  that  these  anomalies  show  up  in  the  magnetometer  data 
but  not  the  EM61  Mk2  data;  this  is  discussed  in  detail  in  section  4.  The  same  is  true  for  area2c 
below,  where  the  density  of  anomalies  typical  of  compact  ferrous  objects  is  even  less. 
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Figure  36:  area2b  near  SORT,  13.2  acres,  mag,  +/-  50nT 
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Figure  37:  area2c  near  SORT,  15.8  acres,  EM,  gate3,  +/-  50mV 
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Figure  38:  area2c  near  SORT,  13.2  acres,  mag  +/-  50nt 


Area3a  was  located  south  of  the  SORT.  185  anomalies  were  analyzed,  85  of  which  were  flagged 
as  Likelihood  2. 
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Figure  39:  area3a  SORT,  15.8  acres,  EM,  gate3,  +/-  50mV 


Area3b  was  located  near  the  New  Demolition  target.  This  was  a  more  cluttered  area  than  had 
been  seen  before,  with  a  large  number  of  anomalies  in  both  the  magnetometer  and  the  EM61 
data.  A  pipeline  is  clearly  visible  in  the  magnetometer  data,  but  does  not  show  up  in  the  EM61 
data.  A  total  of  243  anomalies  were  analyzed  in  area3b,  68  of  which  were  flagged  as  Likelihood 
2. 
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Figure  41:  area3b  near  new  demolition  target,  6.2  acres,  EM,  gate3,  +/-  50mV 
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Figure  42:  area3b  near  new  demolition  target,  6.2  acres,  mag,  +/-  50nT 
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Area4a  was  located  in  the  northwest  comer  of  the  site,  south  of  target  N3.  The  consistency  of  the 
size  of  the  anomalies,  particularly  in  the  magnetometer  data,  is  striking.  The  majority  of  these 
were  parts  of  100  lb  practice  bombs.  78  anomalies  were  analyzed  of  which  39  were  Likelihood  2. 
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Figure  43:  area4a  near  target,  N3, 5.5  acres,  EM,  gate3,  +/-  50mV 
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Figure  44:  area4a  near  target,  N3,  5.5  acres,  mag,  +/-  50nT 

Area4b  was  located  just  north  of  target  N3,  in  the  extreme  northern  comer  of  the  site.  The 
rippling  visible  in  both  the  magnetometer  and  EM  data  images  is  due  to  the  proximity  of  the  area 
to  the  power  line  that  runs  along  the  northern  edge  of  the  site.  The  magnetometer  data  become 
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unusable  right  near  the  power  line.  The  EM61  data,  however,  while  noisy,  are  still  quite  usable. 
This  is  an  example  of  the  inherent  advantage  of  a  concurrent  multisensor  system.  102  anomalies 


Figure  46:  area4b  near  target,  N3, 5.5  acres,  mag,  +/-  50nT 


Area4c  was  located  near  the  SORT  target.  66  anomalies  were  analyzed,  19  of  which  were 
declared  as  Likelihood  2. 
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Figure  47:  area4c  near  oil  refinery  target,  14  acres,  EM,  gate3,  +/-  50mV 
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Figure  48:  area4c  near  oil  refinery  target,  14  acres,  mag,  +/-  50nT 


66 


Area4d  was  located  near  the  New  Demolitions  target.  This  was  the  most  active  of  all  the  100% 
geophysical  survey  areas,  with  a  very  high  anomaly  density  in  both  the  magnetometer  and  the 
EM61  data,  indicating  that  most  of  the  signals  were  most  likely  not  geology  but  metallic  in 
origin.  382  anomalies  were  analyzed,  of  which  87  were  declared  as  Likelihood  2. 
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Figure  49:  area4d  near  new  demolition  target,  3  acres,  EM,  gate3,  +/-  50mV 
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Figure  50:  area4d  near  new  demolition  target,  3  acres,  mag,  +/-  50nT 
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Area4e  was  located  in  the  southern  part  of  the  site.  Of  all  of  the  100%  geophysical  survey  areas, 
area4e  was  the  most  geologically  cluttered.  This  geological  activity  is  evident  from  the  number 
of  misshapen  dipoles  in  the  magnetometer  data  that  have  no  corresponding  anomaly  in  the  EM61 
data.  A  total  of  363  anomalies  were  analyzed,  of  which  41  were  declared  as  Likelihood  2. 
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Figure  51:  area4e  on  south  site,  20.3  acres,  EM,  gate3,  +/-  50mV 
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Figure  52:  area4e  on  south  site,  20.3  acres,  mag,  +/-  50nT 
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Area4f  was  also  on  the  southern  part  of  the  site.  141  anomalies  were  analyzed,  of  which  7  were 
declared  as  Likelihood  2. 
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Figure  53:  area4f  on  south  site,  15.7  acres,  EM,  gate3,  +/-  50mV 
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Figure  54:  area4f  on  south  site,  15.7  acres,  mag,  +/-  50nT 


The  anomaly  densities  for  areas  3  a  and  3b  and  4a  through  4f  do  not  reveal  a  specific  trend  in  the 
way  that  those  of  areas  la,  lb,  and  lc,  and  2a,  2b  and  2c  did,  since  only  these  later  areas  were 
designed  to  directly  measure  the  falloff  of  target  density.  We  include  the  anomaly  densities  of  the 
other  areas  here  for  completeness. 


Table  11:  Anomaly  Density  of  Non-Radially  Situated  Areas 


number  of 
likelihood  2 
anomalies 

near  target 

acreage 

anomaly 
density 
(number 
per  acre) 

Area3a 

85 

SORT 

15.8 

5.4 

Area3b 

68 

New  Demolition 

6.2 

11.0 

Area4a 

39 

N3 

5.5 

7.1 

Area4b 

38 

N3 

8 

4.8 

Area4c 

19 

SORT 

14 

1.4 

Area4d 

87 

New  Demolition 

3 

29.0 

Area4e 

41 

South  Site 

20.3 

2.0 

Area4f 

7 

South  Site 

15.7 

0.4 
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4.3.4  Analysis  of  100%  Geophysical  Survey  Data 

The  ground  truth  file  (“kirtland  all  areas  9-22-06.xls”)  distributed  by  the  ESTCP  Program  Office 
contains  an  analysis  of  the  percentage  of  the  778  dug  anomalies  at  the  Kirtland  PBR  that  were 
assigned  to  each  class.  The  breakdown  is  intact  ordnance  (1%),  ordnance-related  scrap  (73%), 
non-ordnance-related  scrap  (8%),  and  no-find  (geo)  (18%).  The  18%  no-find  rate  is  higher  than 
we  expected,  so  the  geophysical  data  underlying  these  no-find  locations  were  examined  closely. 
All  the  sample  images  below  are  from  area4e,  as  this  was  the  most  geologically  active  of  the  14 
100%  geophysical  survey  areas  and  had  the  highest  number  of  no-finds.  However,  the  analysis 
was  performed  not  just  in  area4e  but  in  all  14  areas. 

As  part  of  the  Kirtland  survey  in  autumn  of  2005,  the  ground-based  data  from  all  14  of  the  100% 
geophysical  survey  areas  were  examined.  A  total  of  2502  anomalies  were  analyzed  in  these 
areas,  of  which  a  total  of  705  were  declared  as  Likelihood  2.  Anomaly  lists  containing  these 
anomalies  were  submitted  to  the  Program  Office.  The  Final  Kirtland  Validation  Plan  distributed 
by  the  Program  Office  makes  distinctions  on  an  area-by-area  basis  as  to  which  geophysical  data 
sets  (helicopter,  ground-based,  or  both)  were  used  to  determine  which  anomalies  to  dig  in  which 
area.  However,  for  the  sake  of  this  false  alarm  analysis,  we  regard  these  equally.  That  is,  any  dug 
anomaly  whose  ground  truth  was  located  in  one  of  the  14  100%  geophysical  survey  areas 
covered  with  VSEMS  was  included  in  this  analysis.  There  were  289  such  anomalies.  Of  these, 
251  were  metallic  objects  of  some  sort,  and  39  are  listed  in  the  dig  sheet  as  “geology  (no-find).” 
By  this  calculation,  the  probability  of  false  alarm  for  digging  and  not  finding  metal  is  13.5%. 

It  should  be  noted  that  although  the  dig  sheet  does  include  digger’s  comments  that  attempt  to 
distinguish  between  the  distinct  categories  of  “hot  rock,”  “geo  layer,”  and  “no  contact,”  all  three 
of  these  are  encapsulated  by  the  dig  sheet  in  the  single  category  “geology  (no-find),”  and  the  dig 
usage  is  inconsistent,  sometimes  using  several  of  the  terms  for  one  item.  For  this  reason,  we  did 
not  attempt  to  split  out,  for  example,  hot  rocks  from  no  contacts,  and  instead  regarded  all 
“geology  (no-find)”  objects  equally.  The  sole  exceptions  are  the  two  truly  explainable  no-finds 
caused  by  the  analyst’s  flagging  the  centers  of  two  of  the  “crop  circles”  as  Likelihood  2  targets; 
these  have  already  been  excluded  from  the  list  of  39. 

We  wished  to  see  how  we  performed  in  our  original  heuristic  classification  of  anomalies  as 
likely  geology.  Further,  we  were  interested  in  what  the  probability  of  false  alarm  would  fall,  to 
with  the  application  of  two  post-survey  geologic  screening  strategies. 

Because  VSEMS  concurrently  collects  both  magnetometer  and  EM61  data,  it  provides  the 
potential  for  an  experienced  operator  to  exclude  many,  if  not  most,  geologic  anomalies. 
However,  note  that  there  is,  at  present,  no  objective,  algorithmic,  turn-key  mechanism  to  analyze 
both  sensor  streams  and  determine  if  the  anomaly  of  interest  is  likely  geology  or  MEC.  At 
present,  this  operation  is  performed  heuristically  by  the  analyst.  At  the  beginning  of  the  Kirtland 
survey,  we  felt  that  a  anomaly  with  strong,  clear,  round  dipolar  lobes  in  the  magnetometer  data,  a 
good  fit  to  a  point  dipole,  but  no  discemable  EM61  signature  was  most  likely  an  MEC  object  that 
had  penetrated  too  deep  for  detection  by  the  EM61  (the  Kirtland  PBR  was,  after  all,  a  bombing 
range).  After  examining  the  Kirtland  ground  truth,  however,  the  evidence  is  compelling  that,  at 
this  site,  the  “good  mag  signature  / no  EM61  signature”  combination  strongly  connotes  not  a 
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deep  MEC  item,  but  instead  a  geologic  false  positive.  That  is,  the  weird,  irregular-looking 
anomalies  turned  out  (no  surprise)  to  be  geology,  but  so  did  many  of  the  regular-looking  dipoles 
(surprise)  that  lacked  a  confirmatory  EM  signature.  Thus,  one  strategy  is  to  look  at  the  false 
alarm  reduction  gained  by  requiring  a  confirming  EM61  signature.  The  other  is  to  simply 
reexamine  the  magnetometer  signatures  to  see  if  the  analyst  could  have  done  a  better  job 
screening  out  mag  signatures  indicative  of  geology. 

The  false  alarm  analysis  began  with  locating  the  analyzed  target  that  correlated  with  the  dug  no- 
find  location  (the  original  numbering  scheme  was  different).  When  the  anomaly  was  originally 
analyzed,  the  operator  assigned  it  a  heuristic  classification  of  2  if  he  thought  it  was  ordnance¬ 
like,  0  if  he  thought  it  was  geology,  noise,  or  clutter-like,  and  1  for  everything  between.  Our 
understanding  is  that  only  our  Likelihood  2  anomalies  were  considered  for  digging.  Thus,  the 
likelihood  0  anomalies  (anomalies  we  felt  were  likely  geology  or  noise)  were  already  screened 
out.  For  example,  the  figure  below  shows  two  anomalies  in  the  magnetometer  data.  The  left 
anomaly,  anomaly  k-864  (VSEMS  anomaly  74)  was  selected  by  the  analyst  as  Likelihood  2 
because  it  was  dipolar  (though  imperfect)  and  thus  it  was  dug.  When  dug,  it  was  a  no-find.  The 
anomaly  to  its  right  was  selected  by  the  operator  as  Likelihood  0  because  of  its  non-compact 
geometry,  and  thus  was  never  dug. 


Figure  55:  Magnetometer  data  (left,  +/-  50nT)  and  EM61  gate3  data  (right,  +/-  50mV)  from  area4e.  Yellow 
grid  squares  are  50  meters  on  edge.  Anomaly  k-864  (VSEMS  anomaly  74)  was  selected  (and  was  a  no-find), 
but  the  anomaly  to  the  right  was  flagged  as  likelihood  0  due  to  its  clearly  non-dipolar  shape. 


It  should  be  understood  that  the  incorporation  of  some  anomalies  that  were  likely  geology  into 
the  anomaly  list  was  intentional  In  the  Data  Report  for  this  project,  submitted  to  the  Program 
Office  on  December  29th  2006  (immediately  after  analysis  of  the  100%  geophysical  survey 
areas),  the  PI,  Rob  Siegel,  reported: 

"Note  that,  for  these  total  coverage  areas,  the  definition  of 
"Likelihood  2"  has  been  stretched  somewhat  to  include  large 
targets  that  could  conceivably  be  geology.  That  is,  in  the 
transect  data  analysis,  Likelihood  2  was  reserved  for  targets 
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that  the  operator  was  convinced  were  subsurface  metal  of  a  size 
and  shape  consistent  with  MEC,  and  large  strong  anomalies  the 
operator  thought  were  geology  were  listed  as  likelihood  0.  For 
these  [total  coverage]  areas,  the  operator,  in  essence,  is 
willing  to  entertain  the  possibility...  that  he  is  incorrect,  and 
large,  strong,  magnetic  anomalies  that  are  dipolar  but  not 
classically  dipolar  (do  not  have  a  round  positive  and  a  mushroom- 
cap-shaped  negative)  are  listed  as  Likelihood  2,  but  with  a 
comment  indicating  that  they  are  probably  geology.  It  will  be 
instructive  to  see  what  these  turn  out  to  be  after  validation." 

Note  that  all  of  these  intentionally-included  questionable  anomalies  turned  out  to  be  geology  (6 
of  the  20  no-finds  in  area4e  were,  in  fact,  commented  by  the  analyst  as  “big  but  probably 
geology”),  but  that  having  been  said,  other  anomalies  that  turned  out  to  be  geology  were  thought 
to  be  representative  of  compact  ferrous  objects  by  the  analyst.  In  other  words,  a  more  liberal 
classification  of  magnetic  anomalies  as  geology-like  would  exclude  more  false  alarms,  but  that 
would  not  be  sufficient;  it  still  result  in  digging  up  quite  a  bit  of  geology.  This  is  quantified 
below. 

To  attempt  to  quantify  the  number  of  false  alarms,  we  sorted  the  official  dig  sheet  by  ground 
truth  category,  and  examined  the  mag  and  EM61  data  over  the  dug  anomalies  in  the  14  100% 
geophysical  survey  areas  that  were  listed  as  no-find  /  geologic  in  nature.  The  dig  sheet  contains  a 
total  of  39  such  anomalies.  The  breakdown  is  as  follows: 


Table  12:  Breakdown  of  dig  sheet  results  within  14  100%  geophysical  survey  areas  by  category 


Category 

Number 

Explanation 

A 

1 

Analyst  said  it  was  geology,  and  it  was 

B 

15 

Mag  dipole  is  geology-like,  but  analyst  was  overly  conservative 

C 

13 

With  the  benefit  of  hindsight,  the  mag  dipole  isn’t  classically  typical  of  a 
compact  ferrous  object,  but  isn’t  clearly  geology  either 

D 

7 

mag  dipole  resembles  a  compact  ferrous  objects;  no  clear  way  to  tell  it’s 
geology  other  than  the  lack  of  a  confirming  EM61  signature 

E 

3 

Dig  results  said  no-find,  but  there’s  a  confirming  EM61  signature  so  the 
analyst  is  concerned  that  the  real  object  may  have  been  missed 

It  may  seem  surprising  that  there  is  only  one  entry  in  category  A  (“analyst  said  it  was  geology, 
and  it  was”),  but  this  is  because  anomalies  that  the  analyst  felt  were  geology  were  flagged  as 
Likelihood  0,  and  we  were  told  that  only  our  Likelihood  2  anomalies  were  used  as  part  of  the 
multilayer  analysis.  Thus  the  surprise  is  not  that  there  weren’t  more  of  these,  but  how  this  one 
crept  in.  It  is  likely  that  the  helicopter  mag  system  flagged  this  as  an  anomaly  of  interest. 

In  category  B,  as  per  the  above  comment  from  the  Data  Report,  there  were  some  anomalies  that 
the  analyst  thought  were  likely  to  be  geology,  but  were  intentionally  flagged  as  Likelihood  2 
both  to  be  conservative  as  well  as  to  get  verification  via  intrusive  sampling.  Examples  are  shown 
in  the  figure  below.  Anomalies  k-876  and  k-877  (VSEMS  anomalies  251  and  252)  are  distorted 
dipoles  in  the  mag  data,  with  no  confirming  EM  signature,  but  the  operator  felt  that  it  was 
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possible  (though  unlikely)  that  they  were  actual  objects  embedded  in  active  geology,  and  thus 
flagged  them  as  Likelihood  2.  15  of  the  39  no-finds  in  the  100%  geophysical  survey  areas  fell 
into  this  category,  where  if  the  analyst  was  simply  more  conservative,  they  would  be  excluded. 


Figure  56:  Magnetometer  data  (left,  +/-  50nT)  and  EM61  gate3  data  (right,  +/-  50m V)  from  area4e.  Yellow 
grid  squares  are  50  meters  on  edge.  Operator  selected  anomalies  k-876  and  k-877  (VSEMS  anomalies  251  and 
252)  because  of  the  possibility  that  MEC  was  embedded  in  geology.  During  remediation,  both  were  listed  as 
no-find  (geology).  These  could  be  excluded  simply  by  the  analyst  being  more  conservative. 

Skipping  down  to  category  D,  7  of  the  39  dug  no-finds  had  mag  dipole  signatures  that  were 
typical  of  compact  ferrous  objects;  even  with  hindsight,  the  analyst  cannot  say  that,  given  mag 
data  only,  he  would  not  have  selected  them  as  potential  MEC.  A  sample  is  shown  in  the  figure 
below. 
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Figure  57:  Magnetometer  data  (left,  +/-  50nT)  and  EM61  gate3  data  (right,  +/-  50m V)  from  area4e.  Yellow 
grid  squares  are  50  meters  on  edge.  Anomaly  k-884  (VSEMS  anomaly  335)  presents  a  compact  dipolar 
signature  with  tight  round  north-facing  lobes.  Even  with  hindsight,  the  operator  can’t  say  that  it  is  anything 
other  than  typical  of  a  compact  ferrous  object.  Only  the  lack  of  a  confirming  EM  signature  suggests  that  it  is 

geology,  which  apparently  it  is. 

Anomalies  in  category  C,  representing  13  of  the  39  anomalies,  fell  somewhere  between  the  two  - 
that  is,  anomalies  in  this  category  did  not  have  perfectly-formed  mag  dipoles,  but  neither  were 
the  shapes  so  irregular  that  were  clearly  geologic  in  origin.  Examples  of  this  are  shown  in  the 
figure  below  with  anomaly  856,  which  is  dipolar  but  non-ideal.  It  is  not  north-aligned,  its  lobes 
are  somewhat  diffuse,  the  negative  lobe  does  not  wrap  around  the  positive  lobe  with  the  classic 
mushroom-capped  shape. 


Figure  58:  Magnetometer  data  (left,  +/-  50nT)  and  EM61  gate3  data  (right,  +/-  50mV)  from  area4e.  Yellow 
grid  squares  are  50  meters  on  edge.  Anomaly  k-856  (VSEMS  anomaly  70)  presents  itself  in  the  magnetometer 
data  (left)  as  a  weak  dipole  with  somewhat  diffuse,  non-north-aligned  lobes  and  no  EM61  signature. 
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The  significant  thing  is  that,  for  cases  A  through  D,  comprising  36  of  the  39  no-finds  in  the 
100%  geophysical  survey  areas,  none  of  the  mag  dipoles  had  a  confirming  EM61  signature. 
That  is,  these  36  apparent  geologic  false  alarms  could  have  been  screened  out  by  simply 
requiring  the  EM61  to  present  a  confirming  signature. 

In  category  E,  the  remaining  three  no-finds  did  have  a  confirming  EM61  signature.  An  example 
of  this  is  shown  in  the  figure  below.  Anomaly  k-844  presents  itself  in  the  magnetometer  data 
(left)  as  a  strong  dipole  with  well-shaped,  north-aligned  lobes.  The  EM61  gate3  data  (right) 
shows  a  strong,  well-formed,  largely  round  confirming  signature.  The  digger’s  comment  says 
“hot  rock  4’  under  stake.”  There  are  several  possible  reasons  for  this  apparently  anomalous 
behavior.  One  is  that  the  “hot  rock”  was  indeed  hot  to  the  EM61  as  well  as  the  mag  (e.g.,  that 
some,  but  not  most,  of  the  volcanic  geology  on  the  Kirtland  site  does  indeed  present  itself  to  the 
EM61).  The  other  is  that  the  object  actually  producing  the  signal  was  not  found  by  the  digger. 
But  it  is  important  to  note  that  these  three  cases  correspond  to  false  positives.  That  is,  they  would 
still  be  dug;  MEC  would  not  be  left  in  the  ground. 


Figure  59:  Magnetometer  data  (left,  +/-  50nT)  and  EM61  gate3  data  (right,  +/-  50mV)  from  area4e.  Yellow 
grid  squares  are  50  meters  on  edge.  Anomaly  k-844,  listed  by  the  dig  team  as  no-find  (geology),  has  a  strong 
magnetic  dipole  (left)  with  round,  north-aligned  lobes  and  a  very  strong  confirming  EM61  signature  (right), 
making  us  wonder  if  the  dig  team  successfully  relocated  and  dug  the  object  producing  the  signal,  or  if  the 

EM61  signal  actually  is  geologic  in  nature. 

The  probability  of  false  alarm  can  be  recalculated  two  ways.  First,  if  we  re-examine  the 
geophysical  data  and  screen  out  the  anomalies  whose  mag  signature  is  clearly  geology-like 
(anomalies  that  the  analyst  only  included  because  he  was  being  overly  conservative  and  wanted 
to,  in  fact,  generate  digs  for  some  anomalies  that  were  likely  geology),  the  probability  of  false 
alarm  is  not  39/289  but  23/289,  or  about  8%.  This  represents  the  best  that  the  analyst  could’ve 
done  without  taking  advantage  of  the  EM61  data. 

However,  if  we  take  advantage  of  the  position  that  a  confirming  EM61  signature  is  required  to 
rule  out  geology,  then  the  probability  of  false  alarm  falls  to  3/289,  or  about  1%. 

The  four  false  alarm  numbers  are  encapsulated  in  the  table  below. 
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Table  13:  Probability  of  false  alarms  before  and  after  data  reexamination 


Probability  of  false  alarm  (no-finds  ->  probable  geology)  from  original  Program 
Office  Dig  Sheet  (includes  anomalies  from  VSEMS  ground-based  data  plus  other 
data) 

18% 

Probability  of  false  alarm  using  anomalies  only  from  VSEMS  ground-based  data 
on  the  14  100%  geophysical  survey  areas  for  which  there’s  ground  truth  on  the 
Program  Office  Dig  Sheet 

13.5% 

Probability  of  false  alarm  removing  objects  whose  mag  signatures  are  clearly  non- 
dipolar  (the  best  the  analyst  could  do  with  mag-only  data) 

8% 

Probability  of  false  alarm  requiring  a  confirming  signature  in  the  EM61  data 

1% 

We  believe  this  to  be  a  highly  significant  result  that  helps  to  justify  the  use  of  concurrent 
multisensor  technology.  The  Kirtland  PBR  was  a  bombing  range  where  air-dropped  ordnance 
potentially  was  used,  including  potentially  250  lb  bombs,  making  it  the  kind  of  site  where,  if  you 
had  to  choose  one  sensor  (magnetometers  or  EM61s),  you  would  likely  choose  magnetometers 
(as  were  chosen  for  all  of  the  other  WAA  sites).  Simply  by  co-deploying  EM61s  and  requiring  a 
confirmatory  signature,  the  probability  of  false  alarm  could  be  cut  from  13.5%  to  1%.  We  note 
that  this  would  need  to  be  combined  with,  at  minimum,  a  depth  determination  so  that  you 
wouldn’t  exclude  the  very  objects  -  the  deep  UXO  -  that  the  magnetometers  can  detect  but  the 
EM61s  cannot.  There  is  insufficient  data  to  attempt  a  depth-based  cutoff  analysis  with  results 
from  the  Kirtland  PBR,  as  there  were  no  250  lb  bombs  found,  there  were  only  five  100  lb 
practice  bombs  found,  and  the  deepest  was  at  3.1  feet.  Note  that,  had  250  lb  bombs  been  found  at 
the  Kirtland  site,  exclusion-zone  requirements  would  have  precluded  digging. 

4.3.5  Survey  Speed,  EM61  Data  Quality,  and  Productivity 

The  magnetometer’s  custom  sampling  electronics  outputs  data  at  75  Hz,  resulting  in  very  dense 
down-track  mag  data  spacing,  but  the  output  rate  of  the  COTS  EM61  Mk2  is  a  comparatively 
slow  10  Hz,  resulting  in  much  sparser  down-track  data  spacing.  For  transect  surveys,  we 
endeavored  to  keep  speed  below  2  meters  per  second  in  order  to  generate  EM61  data  at  least 
every  20  cm,  but  for  the  100%  geophysical  survey  areas,  we  generally  capped  speed  at  a  lower 
rate  of  one  meter  per  second  to  generate  EM61  updates  at  10  cm  down-track  intervals.  Both  of 
these  speeds  may  strike  some  as  slow  in  comparison  to  survey  speeds  with  magnetometer-only 
systems.  This  fuels  the  perception  that,  when  deploying  VSEMS,  a  productivity  hit  is  taken  in 
order  to  collect  the  EM61  data  -  that  you  could  drive  faster,  and  thus  be  more  productive,  if  you 
collect  magnetometer-only  data. 

Our  position  is  that  productivity  -  acres  per  day  -  is  a  function  of  many  factors.  At  the  Kirtland 
PBR,  as  we  described  above,  the  major  productivity  factor  was  GPS  jamming.  We  feel  that,  in 
theory,  all  things  being  equal,  the  slower  EM  sampling  rate  does  affect  productivity,  but  in 
practice,  all  things  are  rarely  equal,  and  that  other  factors  frequently  dominate  this  one. 

That  having  been  said,  because  we  in  fact  collected  data  over  the  100%  geophysical  survey  areas 
not  only  at  the  “slow”  speed  but  at  the  “fast  speed”  (twice  by  accident  and  once  by  design),  we 
can  see  if  the  higher  speed  creates  an  effect  that  we  can  see  in  our  EM61  data. 
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The  table  below  shows  the  data  density  for  the  fourteen  100%  geophysical  survey  areas.  Due  to  a 
miscommunication  with  the  driver,  the  first  two  areas  surveyed  (la  and  lb)  utilized  the  higher  2 
m/s  survey  speed.  The  last  area  surveyed  (4f)  was  intentionally  traversed  at  higher  speed  due  to 
our  trying  to  complete  the  survey  before  an  engine  oil  leak  felled  the  survey  vehicle.  The  other 
surveys  were  conducted  at  speeds  near  1  m/s. 

Table  14:  Data  density  for  the  fourteen  100%  geophysical  survey  areas 


Data  Density  (points  /  m2) 
mag  em 


areal a 

95 

15 

areal b 

108 

14 

arealc 

195 

26 

area2a 

241 

32 

area2b 

195 

26 

area2c 

192 

25 

area3a 

192 

25 

area3b 

163 

22 

area4a 

179 

24 

area4b 

172 

23 

area4c 

147 

19 

area4d 

151 

20 

area4e 

179 

24 

area4f 

107 

14 

Area2a  is  representative  of  slow  speed  data,  and  area  lb  is  representative  of  high  speed  data.  In 
the  image  below,  a  section  of  EM61  gate3  data  from  area2a,  acquired  at  a  slow  1  meter  per 
second,  is  shown.  The  section  is  intentionally  chosen  to  be  relatively  free  from  obvious 
anomalies  due  to  subsurface  objects.  The  background  appears  as  a  clear,  solid  green  color. 


D3  E3 


D2  E2 


Figure  60:  EM61  gate3  data,  +/-  50  mV.  Yellow  grid  squares  are  50  meters  on  edge.  Background  EM61  gate3 

data  from  arealc  acquired  at  1  m/s  appears  clean 
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Below  we  show  some  representative  time-series  data  from  EM61  gate3  in  arealc.  Statistical 
analysis  of  a  larger  representative  section  shows  the  RMS  variation  to  be  .29  mV. 

25. 0i£ 


7.50 


-10.00 


Figure  61:  Oasis  time-series  plot  of  30  seconds  of  EM61  gate3  background  data  from  area2a.  Like  the  above 

image,  data  are  relatively  clean. 


In  the  next  image,  background  EM61  gate3  data  from  area  lb  acquired  at  a  fast  2  meters  per 
second,  is  shown.  The  image  shows  an  obvious  stippling  of  the  background  that  is  not  present  in 
the  slower  survey  shown  above. 


E5  F5 
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Figure  62:  EM61  gate3  data,  +/-  50  mV.  Yellow  grid  squares  are  50  meters  on  edge.  Background  EM61  data 

from  arealb  acquired  at  1  m/s  appears  slightly  noisy 

The  time-series  data  from  EM61  gate3,  below,  confirms  the  stippling  in  the  image  data. 
Statistical  analysis  of  a  larger  representative  section  shows  the  RMS  variation  to  be  .52  mV. 
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Figure  63:  Oasis  time-series  plot  of  30  seconds  of  EM61  gate3  background  data  from  arealb.  Like  the  above 

image,  data  are  slightly  noisy. 


We  believe  that  what  we  are  seeing  here  has  nothing  to  do  with  data  density,  but  is  the  effects  of 
EM61  array  motion  caused  by  platform  motion.  Although  the  slower  survey  speed  was  intended 
to  create  higher  down-track  data  density,  it  had  an  additional  positive  effect  of  reducing  EM61 
sensor  motion.  As  we’ve  said  above,  the  prototype  fiberglass  towed  platform  used  on  VSEMS 
was  only  intended  to  survive  a  single  fielding  at  APG  in  2002.  In  order  to  survive  subsequent 
real-world  surveys  at  Lowry,  Kirtland,  and  other  sites,  survivability  modifications  were  made. 
These  included  mounting  the  EM61  lower  coils  in  a  porch-swing  configuration  that  allowed 
them  to  swing  back  (rather  than  breaking  off)  if  they  encountered  an  obstacle,  and  the 
incorporation  of  titanium  snowmobile  springs  to  provide  a  simple  suspension.  These  certainly 
improved  the  platform’s  longevity  by  giving  ground-induced  shock  somewhere  to  go,  but 
probably  increased  EM61  sensor  motion.  This  fiberglass  platform  has  since  been  retired  and  has 
been  replaced  with  a  new  carbon-fiber  platform  with  a  wider  track,  a  tuned  suspension,  and  no 
“porch  swing”  for  the  EM61s. 

From  the  above  data,  it  appears  that  the  platform  configuration  used  to  host  the  EM61s  at 
Kirtland  is  not  one  that  would  be  optimal  for  assessing  the  degree  of  site  contamination  with 
small  objects  with  weak  signals,  such  as  20mm  projectiles,  at  high  speed  (though  it  may  be  fine 
at  low  speeds).  It  may  be  difficult  if  not  impossible  to  separate  the  original  question  (what  is  the 
effect  of  speed  and  the  resulting  down-track  data  density  on  EM61  data  quality)  from  what  we 
have  found  (that  speed  appears  to  engender  platform  motion,  which  has  a  negative  effect  on 
EM61  data  quality).  However,  note  that  this  noise  does  not  prevent  strong  signals  from  ringing 
out  very  clearly  in  the  EM61  data,  as  shown  in  the  plot  below.  The  anomaly  in  the  cross-hairs 
corresponds  to  target  k-536,  which  was  M38  bomb  scrap.  The  anomaly  produced  by  gate  3  has  a 
width  above  background  of  approximately  3.5  meters,  and  has  a  full  width  at  half  max  of 
approximately  1.5  meters.  Even  at  the  higher  2  m/s  survey  speed,  this  anomaly  contains  14  data 
points  above  background,  indicating  that,  for  objects  of  this  size,  higher  survey  speeds  can  create 
data  with  sufficient  density  to  allow  object  detection. 
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Figure  64:  EM61  data  over  M38  bomb  scrap  in  arealb  clearly  shows  a  good,  sharp  target  response  even  with 

increased  background  noise  due  to  elevated  survey  speed 

The  speed  at  which  the  survey  is  conducted  should  be  a  function  of  several  things.  Higher  speed 
may  increase  productivity,  but  it  beats  up  survey  equipment.  How  the  speed  affects  EM61  data 
quality  certainly  should  be  considered,  but  rather  than  merely  specify  a  maximum  speed  in  order 
to  meet  a  minimum  EM61  data  density  metric,  the  way  in  which  the  EM61  data  are  to  be  used 
should  be  considered.  If  the  goal  of  using  the  EM61  is  not  to  detect  small  objects  with  weak 
signatures,  but  instead  to  screen  against  geologically-induced  false  alarms,  then  higher  speed 
(and  with  it,  lower  data  densities,  likely  higher  noise,  and  likely  increased  need  for  platform 
maintenance)  can  probably  be  tolerated. 

4.3.6  Inversion  of  EM61  Data  in  100%  Geophysical  Survey  Areas 

In  analyzing  the  transect  data  for  next-day  turnaround,  SAIC’s  existing  mag  dipole  inversion  was 
used,  as  its  speed  and  ease  of  use  enabled  next-day  turnaround  of  results.  The  EM61  data  was 
always  collected,  processed,  and  displayed  (the  analyst  always  views  both  data  sets  during 
analysis,  as  the  presence  or  absence  of  correlating  anomalies  in  both  data  sets  is  important 
information),  but  the  EM61  data  was  not  inverted  as  part  of  the  transect  analysis,  as  both  the 
VSEMS  data  analysis  software  and  Geosoft  Montaj™  lack  EM  inversion  code.  As  we  focused 
on  the  analysis  of  the  100%  geophysical  survey  areas,  however,  the  Program  Office  requested 
that  we  propose  a  method  for  inverting  not  only  the  magnetometer  data,  but  the  EM61  data  as 
well.  We  supplied  the  Program  Office  with  several  options  for  this,  and  the  one  selected  was  to 
use  the  UxAnalyze  module  for  Oasis  being  developed  by  AETC  and  Geosoft  with  funding  from 
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ESTCP.  To  accomplish  this,  we  received  training  in  Oasis  while  in  Albuquerque,  and  traveled  to 
AETC  for  training  on  UxAnalyze. 

After  the  data  acquisition  was  complete,  we  began  using  Montaj™  to  analyze  the  100% 
geophysical  survey  area  data.  It  was  our  assumption  that  we  would  adopt  the  approach  used  by 
many  UXO  geophysicists  and  use  the  automatic  anomaly  picking  feature  in  Montaj ’s  UxDetect 
module  to  pick  anomalies  similar  to  those  picked  manually  from  the  transect  data.  Then  we 
assumed  we  would  feed  this  anomaly  list  to  UxAnalyze.  However,  a  number  of  things  made  this 
problematic. 

Firstly,  in  order  to  perform  automatic  anomaly  selection  on  magnetometer  data,  one  first  must 
create  analytic  signal  (essentially  the  two-dimensional  spatial  derivative)  to  convert  the  positive¬ 
negative  dipolar  anomalies  into  unipolar  anomalies.  However,  since  analytic  signal  has  a  spatial 
component,  there  is  no  direct  correspondence  between  the  threshold-based  manual  picking 
employed  on  the  transect  data  (everything  over  10  nT  and  over  a  certain  spatial  size)  and  the 
values  picked  in  the  analytic  signal  data.  Thus  it  was  difficult  using  this  technique  to  pick 
anomalies  in  a  method  that  is  consistent  with  what  was  already  done  with  the  transect  data. 

Second,  when  using  UxDetect  to  automatically  pick  anomalies,  these  anomalies  must  be  vetted 
by  a  trained  operator.  Inappropriate  picks  must  be  deleted,  additional  picks  must  be  added,  and 
clusters  of  picks  that  represent  the  same  object  must  be  merged.  There  are  utilities  to  perform 
these  steps  in  Oasis,  but  they  are  time-consuming  enough  that  we  decided  to  stick  with  hand¬ 
picking  anomalies  in  our  own  software. 

Third,  the  familiarity  we  have  with  our  own  data  and  how  it  is  processed,  gridded,  and  viewed  in 
our  own  software,  is  valuable  in  the  overall  context  of  expert  analysis.  One  example  of  this 
involves  spurious  data.  After  processing  and  viewing  the  VSEMS  magnetometer  data  from  one 
of  the  100%  geophysical  survey  areas  in  our  own  software,  we  brought  it  into  Montaj™,  created 
analytic  signal,  and  used  UxDetect  to  automatically  select  anomalies.  When  sanity-checking  the 
results,  numerous  picks  had  been  automatically  selected  that  overlaid  strong,  sharp  anomalies 
that  we  did  not  remember  seeing  in  the  data.  We  viewed  the  data  in  our  own  software  to  see  what 
these  signals  were,  and  they  were  coming  from  spurious  readings  on  the  order  of  50  nT  that  had 
squeaked  through  our  median  filter,  which  normally  rejects  spikes  over  100  nT.  In  our  own 
software,  when  gridded,  these  spurious  readings  created  small  square  interpolation  artifacts  that 
are  ignored  by  expert  eyeballs,  but  in  Oasis  they  were  iteratively  gridded  until  they  created 
spectacularly  strong  anomalies. 

For  all  of  the  above  reasons,  we  decided  to  hand-pick  anomalies  from  the  100%  geophysical 
survey  areas  using  our  own  software  in  a  method  as  close  as  possible  to  that  which  was  used  in 
the  transect  data,  and  to  then  pass  these  locations  to  the  EM  inversion  routines  in  UxAnalyze. 
Note  that,  for  these  100%  geophysical  survey  areas,  the  definition  of  “Likelihood  2”  was 
stretched  somewhat  to  include  large  anomalies  that  could  conceivably  be  geology,  as  described 
above. 

In  consultation  with  Dr.  Dean  Keiswetter  at  AETC  and  the  ESTCP  Program  Office,  it  was 
decided  that  the  “Likelihood  2”  anomalies  presented  a  good  candidate  set  for  UxAnalyze  due  to 
their  generally  strong  signatures.  This  corroborates  our  experience,  as  during  our  foray  into 
UxAnalyze,  we  discovered  that  many  anomalies  with  visible  anomalies  above  threshold  failed  to 
converge  because  the  signal  was  too  weak  or  because  they  did  not  contain  enough  data  to 
generate  high-quality  results. 
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Because  of  delays  in  obtaining  a  qualified,  tested  copy  of  UxAnalyze  that  correctly  models  our 
EM61  data  as  an  array  with  one  large  transmitter  and  five  individual  receivers,  Dr.  Keiswetter 
performed  the  analysis  himself.  He  took  our  Likelihood  2  anomaly  picks,  chose  polygons  that 
wholly  surround  the  objects,  and  ran  the  EM  inversion.  We  used  his  results  to  perform  a  visual 
correlation  of  the  depth  estimates  from  mag  and  EM  inversion.  Three  examples  are  shown. 

The  figure  below  shows  the  depth  estimates  for  the  Likelihood  2  anomaly  picks  in  area4a. 
Correlation  of  depth  results  from  the  two  sensors  is  generally  good,  with  the  exception  of  the 
outlier  caused  by  the  EM-generated  depth  from  point  32.  This  was  item  k-187,  which  according 
to  the  dig  sheet,  was  a  no-find.  Including  this  outlier,  the  r-squared  correlation  (RSQ,  or  the 
proportion  of  the  variance  in  the  second  array  attributable  to  the  variance  in  the  first  array) 
between  the  mag  and  EM  depths  is  0.28  and  the  covariance  is  0.53.  Excluding  this  outlier 
changes  the  results  only  slightly;  the  RSQ  increases  to  0.30  and  the  covariance  increases  to  0.54. 
The  mag  and  EM-derived  depths,  plotted  both  against  item  number  as  well  as  against  each  other, 
are  shown  in  the  next  two  figures,  and  the  anomaly  data  for  item  k-187  is  shown  in  the  third 
figure. 


Mag  and  EM  Depth,  Area4a 


— Mag  Depth 
— EM  Depth 


Figure  65:  Depths  derived  for  Likelihood  2  anomalies  from  mag  (blue)  and  EM  (magenta)  inversion  in  area4a 
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EM  Depth  Versus  Mag  Depth,  Area  4A 


Mag  Fit  Depth  (meters) 


Figure  66:  EM-derived  and  mag-derived  depths  plotted  against  each  other  for  Area4a 


Figure  67:  Magnetometer  data  (left,  +/-  50nT)  and  EM61  gate3  data  (right,  +/-  50mV)  from  area4a.  Yellow 
grid  squares  are  50  meters  on  edge.  Item  k-187  on  the  right  side  of  each  image,  a  no-find,  caused  the  outlier  in 

the  depth  comparison  chart  above. 

In  the  figure  below,  the  mag-derived  and  EM-derived  depths  for  area4e,  a  very  geologically- 
active  area,  are  shown.  The  EM  depth  fails  to  converge  on  six  occasions,  correlating  (like  the 
example  above)  with  no-find  results  in  the  dig  sheet,  and  anomalies  that  exist  only  in  the 
magnetometer  data.  Not  surprisingly,  the  EM  fit  coherence  is  poor  for  these  six  outliers  (0.16, 
0.001,  0.0,  0.003,  0.201,  and  0.018).  The  magnetometer  depth  fails  to  converge  on  the  third  point 
(where  the  EM  also  fails  to  converge).  This  is  target  k-846,  shown  in  the  two  images  below.  Due 
to  the  outliers,  the  results  are  not  well-correlated;  the  RSQ  correlation  between  the  mag  and  EM- 
derived  depth  estimates  is  0.0007  and  the  covariance  is  0.028.  With  the  outliers  removed,  the 
RSQ  increases  substantially  to  0.12  and  the  covariance  to  0.35.  The  mag  and  EM-derived  depths, 
plotted  both  against  item  number  as  well  as  against  each  other,  are  shown  in  the  next  two  figures, 
and  anomaly  data  showing  item  k-846  is  shown  in  the  third  figure. 
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Depth,  Meters 


Mag  and  EM  Depth,  Area4e 


♦  Mag  Depth 
—■—EM  Depth 


Figure  68:  Depths  derived  for  Likelihood  2  anomalies  from  mag  (blue)  and  EM  (magenta)  inversion  in 

Area4e 
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EM  Depth  Versus  Mag  Depth,  Area  4E 


Mag  Fit  Depth  (meters) 


Figure  69:  EM-derived  and  mag-derived  depths  plotted  against  each  other  for  Area4e 


Figure  70:  Magnetometer  data  (left,  +/-  50nT)  and  EM61  gate3  data  (right,  +/-  50mV)  from  area4e.  Yellow 
grid  squares  are  50  meters  on  edge.  Item  k-846,  a  very  faint  positive  swell  in  the  magnetometer  data  and 
nothing  in  the  EM61  data,  was  a  no-find,  and  caused  the  first  outlier  in  the  depth  comparison  chart  above. 

The  other  outlier  in  the  mag  data  was  caused  by  item  k-883,  a  geologic  no-find.  This  pick  was 
caused  by  an  anomaly  in  the  magnetometer  data  that  was  highly  geology-like  but  looked  like  it 
was  possible  (though  unlikely)  that  there  was  an  object  embedded  in  the  geology  as  well.  Both 
the  mag  and  EM  inversions  failed  to  converge  -  the  mag  because  the  signature  did  not  well- 
resemble  and  dipole,  and  the  EM  because  there  was  essentially  no  signal  there  (see  the  figures 
below). 
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Figure  71:  Magnetometer  data  (left,  +/-  50nT)  and  EM61  gate3  data  (right,  +/-  50mV)  from  area4d.  Yellow 
grid  squares  are  50  meters  on  edge.  Item  k-883  (lower  left),  a  no-find  in  the  lower  left  of  each  image,  caused 
the  outlier  in  the  depth  comparison  chart  above.  The  magnetometer  data  is  non-dipolar;  there  is  virtually  no 

signal  in  the  EM  data. 


Other  EM-derived  depths  from  analyses  that  failed  to  converge  in  Area4e  correlate  to  no-find 
targets  where  there  was  little  or  no  EM  signal. 

5  Cost  Assessment 

5.1  Cost  Reporting 

A  by- task  cost  of  performing  the  demonstration  of  VSEMS  at  the  Kirtland  PBR  consists  of  the 
following: 

.  The  cost  of  reinforcing  the  proof-of-concept  fiberglass  platform  to  help  it  survive  the  survey 

.  The  cost  of  mobilization/demobilization  (driving  the  tractor/trailer  to  and  from  Albuquerque) 

.  The  cost  of  a  five- week  and  a  three- week  survey  stint  in  Albuquerque 
.  The  cost  of  analyzing  the  data  back  at  SAIC 
.  The  cost  of  project  management  and  report  writing 

The  actual  breakdown  by  task  is  below.  Mob/demob  includes  driving  the  tractor/trailer  from 
Newton  MA  to  Albuquerque  NM  and  back,  plus  travel  for  the  crew  for  the  two  separate  mobs. 
The  Survey  task  includes  the  four-man  crew  onsite  for  40  days,  plus  all  related  survey  ODCs. 
The  Analysis  task  includes  the  time  spent  analyzing  the  100%  geophysical  survey  data  after  the 
actual  on-site  survey.  It  also  includes  the  costs  of  training  in  Geosoft  Montaj™  and  AETC’s 
UxAnalyze  plug-in,  and  the  time  spent  analyzing  EM61  data.  The  Project  Management  task 
includes  the  site  visit,  all  meetings  and  presentations,  classical  project  management,  and 
reporting.  S AIC’s  VSEMS  already  includes  the  tow  vehicle,  towed  platform,  magnetometers, 
EM61s,  GPS,  and  computers.  As  such,  none  of  these  items  needed  to  be  purchased  for  this 
project.  An  UXO  tech  escort  from  EOTI  was  provided  by  the  ESTCP  program  Office  and  acted 
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as  the  fourth  member  of  the  field  crew;  a  cost  estimate  for  his  presence  is  included  in  the  costing 
below. 


Table  15:  Breakdown  of  Actual  Survey  Costs 

Reinforce  Platform 

$  11,458.06 

Mob/Demob 

$  27,507.75 

Survey 

$  226,366.58 

Analysis 

$  34,436.61 

Project  Management 

$  66,063.95 

Total 

$  365,832.95 

Number  of  Acres 

5000 

Cost  Per  Acre 

$72 

5.2  Cost  Analysis 
5.2.1  Cost  Comparison 

There  are  no  other  concurrent  mag/EM61  vehicular  systems.  The  other  vehicle-towed  arrays 
which  have  non-simultaneously-deployed  EM61  and  magnetometer  platforms  are  1)  the  original 
NRL-fielded  MTADS,  and  2)  the  Blackhawk  (now  Zapata)-fielded  MTADS.  The  NRL  MTADS 
technically  is  not  a  commercially  available  system  but  a  system  for  scientific  study,  and  is 
usually  fielded  by  a  large  crew  of  scientists  and  engineers  on  jobs  intended  to  showcase  the 
system’s  ability  to  collect  discrimination-quality  data.  The  Blackhawk  (Zapata)-fielded  MTADS 
was  intended  to  be  the  commercially-available  version  of  the  NRL-developed  MTADS;  it  is 
uncertain  if  it  is  still  in  use.  Members  of  the  National  Association  of  Ordnance  Contractors 
(NAOC)  with  vehicle-towed  EM61-only  arrays  include  Parsons,  Sky  Research,  USA 
Environmental,  Weston  Geophysics,  Naeva,  ARM,  Shaw,  Tetra  Tech,  SAIC  (a  different  division 
than  the  one  fielding  VSEMS),  and  UXB.  We  are  not  familiar  with  every  one  of  these  systems, 
but  we  know  how  difficult  it  is  to  correctly  deal  with  the  system  timing  issues  necessary  to 
collect  correctly  geo-located  data.  In  order  to  account  for  the  latency  of  EM61  data  (the  fact  that 
it  is  “old”  when  it  comes  out  of  the  box)  and  correctly  synchronize  it  with  geolocation  data,  it  is 
necessary  to  time-stamp  the  data  as  it  arrives  at  the  computer,  and  then  to  perform  a  latency 
correction  by  time-shifting  the  data.  To  do  this,  the  timestamps  must  be  accurate.  Some 
contractors  merely  set  the  PC  clock  to  the  Universal  Time  Coordinate  (UTC)  time  derived  from 
the  GPS.  However,  the  drift  of  the  PC  clock,  combined  with  the  coarse  66ms  precision  of 
reading  this  clock  under  Microsoft  Windows,  make  this  a  fundamentally  inaccurate  way  to 
collect  EM61  data.  Data  sets  acquired  by  this  method  often  have  anomalies  with  “chevron” 
artifacts  because  the  anomalies  are  shifted  in  the  direction  of  travel  due  to  these  timing  issues. 
Some  contractors  use  the  commercial  data  acquisition  software  MagLog,  but  that  does  not  solve 
this  timing  problem,  although  MagLog  combined  with  a  commercial  GPS  timing  card  does. 

The  only  NAOC  members  with  vehicle -towed  magnetometer-only  arrays  are  Sky  Research  and 
ARM  Group.  The  smaller  number  of  towed  magnetometer  systems  is  due  to  several  factors, 
including  the  preponderance  of  statements  of  work  from  the  US  Army  Corps  of  Engineers  that 
mandate  use  of  an  EM61,  the  historical  reliance  of  towed  magnetometer  arrays  on  expensive 
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custom  vehicles  with  low  magnetic  signatures  (this  has  been  evaluated  in  ESTCP  project  MM- 
0605,  which  has  concluded  that  there  are  commercial-off-the-shelf  vehicles  that  work  nearly  as 
well  for  towed  array  magnetometry  as  the  custom  vehicles),  and  the  necessity  of  a  well- 
engineered  system  due  to  the  sensitivity  of  magnetometers  to  any  nearby  ferrous  metal,  including 
the  beads  in  the  tires  on  the  platform.  Our  understanding  is  that  both  the  Sky  and  the  ARM 
system  are  well-engineered  systems  that  pay  appropriate  attention  to  timing  and  signature  issues 
and  generate  high-quality  data. 

Since  VSEMS  uses  COTS  EM61s  and  total  field  magnetometers,  there  is  little  about  VSEMS’ 
data  streams  that  individually  distinguish  them  from  mag  or  EM61  data  from  properly- 
synchronized  data  acquired  by  the  above  contractors.  Although  these  sensors  have  a  broadly 
overlapping  detection  envelope,  the  nod  generally  goes  to  EM61s  for  sites  where  the  objects  of 
interest  are  small  (20mm  and  40mm),  even  though  the  EM61’s  10  Hz  output  rate  limits  the 
survey  speed.  Conversely,  for  the  WAA  objectives  of  detecting  extent  of  bombing  targets 
contaminated  with  air-dropped  munitions,  magnetometers  were  the  sensor  of  choice,  even 
though  the  magnetometer’s  response  to  geology  can  limit  the  interpretability  of  the  data. 
Geology  was  not  expected  to  be  a  problem  at  the  Kirtland  PBR,  but  it  was.  The  analysis  in 
section  4.3.4  shows  that  the  EM61  was  unaffected  by  this  geology,  and  since  VSEMS  was  driven 
slowly  in  order  to  collect  high-quality  EM61  data  anyway,  and  since  there  were  no  deep  mag- 
only  objects  recovered  at  Kirtland,  one  could  argue  that  an  EM61-only  survey  by  a  commercial 
contractor  with  a  well-synchronized  array  would  be  sufficient.  However,  the  absence  of  deep 
mag-only  objects  is  likely  a  function  of  the  lack  of  digging  of  the  target  centers  themselves. 
Indeed,  finding  a  live  250  lb  bomb  would’ve  proven  problematic  at  Kirtland,  as  such  a  discovery 
probably  would’ve  necessitated  closure  of  the  Double  Eagle  airport. 

Similarly,  note  that  the  vehicular  surveys  at  the  other  Wide  Area  Assessment  sites  were  all 
performed  by  the  MTADS  magnetometer  platform.  At  the  Victorville  PBR  Y  site,  the  system 
encountered  unexpected  magnetic  geology,  requiring  additional  surveying  with  man-portable 
EM61  equipment.  Use  of  simultaneous  mag/EM  at  this  site  would  have  concurrently  acquired 
this  EM61  data  and  obviated  the  need  for  a  separate  EM61  survey. 

5.2.2  Cost  Basis 

The  cost  basis  included  a  crew  of  four. 

5.2.3  Cost  Drivers 

The  largest  cost  driver  is  the  time  spent  actually  collecting  survey  data.  This  is  affected  by  many 
things,  including  downtime,  site  access,  site  geometry,  and  weather.  A  secondary  cost  driver  is 
the  vehicular  hospitability  of  the  survey  site  and  the  resulting  forward  rate  of  advance.  A  list  of 
factors  affecting  cost  and  performance  were  included  in  section  2.3  of  this  report. 

5.2.4  Life  Cycle  Costs 

Historically,  deployment  of  the  VSEMS  commercial  UXO  survey  work  has  included  a 
$1950/day  rental  charge.  This  charge  was  originally  used  to  amortize  the  internal  development  of 
the  original  GEO-CENTERS  “STOLS”  system  that  VSEMS  grew  out  of,  and  thus  represented 
the  life  cycle  cost.  Presently,  the  daily  rental  charge  is  typically  used  to  maintain  the  system  and 
repair  broken  sensors  (the  total  field  magnetometers  are  now  1 5  years  old  and  sometimes  die  at 
the  rate  of  one  per  week,  necessitating  a  $5000  repair).  The  daily  rental  charge  is  typically  waved 
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for  a  project  that  is  contracted  through  the  Cooperative  Research  and  Development  Agreement 
(CRADA)  that  SAIC  has  with  CEHNC  (that  is,  waiving  the  rental  charge  is  SAIC’s  contribution 
to  the  CRADA).  However,  the  Wide  Area  Assessment  project  was  not  contracted  through  the 
CRADA,  and  thus  the  proposal  included  the  daily  rental  charge. 

6  Implementation  Issues 

6.1  Environmental  Checklist 

Other  than  the  emissions  created  by  the  tow  vehicle’s  gasoline-powered  internal  combustion 
engine,  no  residuals  or  pollutants  are  produced  by  the  system. 

6.2  Other  Regulatory  Issues 

Because  the  technology  involves  combining  the  two  sensors  most  validated  against  UXO  for 
digital  geophysical  mapping  -  total  field  magnetometers  and  EM61  pulsed  induction  coils  - 
there  are  no  specific  regulatory  issues  above  those  that  apply  to  all  DGM  data.  Any  applicable 
regulatory  issues  involve  detection  and  discrimination  systems  of  all  kinds  (i.e.,  how  clean  is 
clean,  etc)  and  are  not  specific  to  this  project  or  technology. 

6.3  End-User  Issues 

The  envisioned  end-user  is  the  Army  Corps  of  Engineers  Huntsville  (CEHNC)  and  the 
contractors  that  they  employ  for  UXO  site  assessment  and  cleanup.  The  applicability  of  the 
technology  is  largely  a  function  of  the  vehicular  navigability  and  GPS  coverage  on  a  particular 
site;  these  are  factors  that  apply  to  any  towed  vehicular  sensor  array,  not  just  VSEMS. 

Because  the  technology  involves  combining  the  two  sensors  most  validated  against  UXO  for 
digital  geophysical  mapping  -  total  field  magnetometers  and  EM61  pulsed  induction  coils  - 
there  are  no  specific  end-user  issues  above  those  that  apply  to  all  DGM  data.  The  interleaving 
concurrent  multisensor  hardware  used  in  this  project  was  developed  under  ESTCP  Project  MM- 
0208,  and  completed  dem-val  under  that  project.  Since  then,  the  VSEMS  system  was 
successfully  used  to  survey  nearly  500  acres  of  commercial  and  government  MEC,  OE  and 
HTRW  sites  before  being  deployed  to  the  Kirtland  PBR.  There  appear  to  be  no  end-user  issues 
stemming  from  the  interleaving  technology. 
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Appendix:  Test  Strip  Results 

The  figures  below  plot  the  response  for  the  15  items  in  the  test  strip.  For  all  15  figures,  the 
horizontal  axis  represents  the  individual  strip  tests  taken  morning  and  afternoon  during  the  eight- 
week  survey.  The  magnetometer  data  (blue)  in  nT,  the  fit  results  to  magnetometer  data  (magenta) 
in  nT,  and  EM61  gate  3  (yellow)  in  mV  are  plotted  on  the  same  chart.  Obvious  outliers  have 
been  removed.  As  said  in  the  body  of  the  document,  we  attribute  the  variation  to  the  motion  of 
the  sensors  on  the  towed  platform. 
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Test  Strip  Item  2  (37mm  simulant  at  25cm) 


Test  Strip  Item  3  (shotput  at  25cm) 


— raw  mag 
—■—fit  mag 
raw  em 
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Test  Strip  Item  4  (105mm  at  40cm) 


Test  Strip  Item  5  (105mm  at  40cm) 
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Test  Strip  Item  6  (81mm  at  27cm) 


♦  raw  mag 
—■—fit  mag 
raw  em 
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